Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

341
Views
C Python Module with unusual memory usage

I am very new to Python C-API and module creation. I tried to create a c-hash python module. I use python 3.4.3 and TDM-gcc (64bit) 4.9.2 for the compilation on Windows.

Here my code:

// hash_mod.c
#include <Python.h>

unsigned long _hash(unsigned char const* str)
{
    unsigned long hash = 5381;
    int c;
    int i;

    i = 0;
    while (str[i] != '\0')
    {
        c = str[i];
        hash = ((hash << 5) + hash) + c;
        ++i;
    }

    return hash;
}

static PyObject*
hash_hash(PyObject* self, PyObject* args)
{
    unsigned char const* str;

    if (!PyArg_ParseTuple(args, "s", &str))
        return NULL;

    return PyLong_FromUnsignedLong(_hash(str));
}

static PyMethodDef HashMethods[] = {
    {"hash", hash_hash, METH_VARARGS, "String Hash"},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef HashModule = {
   PyModuleDef_HEAD_INIT,
   "hash",
   NULL,
   -1,
   HashMethods,
   NULL,
   NULL,
   NULL,
   NULL
};

PyMODINIT_FUNC
PyInit_hash(void)
{
    return PyModule_Create(&HashModule);
}

The setup.py:

# setup.py
from distutils.core import setup, Extension

module1 = Extension('hash', sources = ['hash_mod.c'])

setup (name = 'Hash',
        version = '1.0',
        description = 'String Hash',
        ext_modules = [module1])

The compilation works well but when I try to import my hash module in the interpreter, my memory make a big jump, more than 2Go for the python.exe process.

Here's a picture of my task manager showing memory usage:

  1. >>> import hash
  2. The import hash finish
  3. Exit python interpreter

After the import finish I can use my module and it works well but the memory seems a bit high.

It seems for me the PyModule_Create does a really big memory allocation. But I am pretty sure this doesn't happen in other module.

Did I miss something ?

Edit:

When I already use a lot of RAM (more than 2.5Go / 4Go), I get this error:

>>> import hash
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>>
over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

As Python uses a garbage collector it doesn't need to release memory at any particular time and you are probably seeing optimisation in action.

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!