Python is passing 32bit pointer address to C functions

Question

I would like to call my C functions within a shared library from Python scripts. Problem arrises when passing pointers, the 64bit addresses seem to be truncated to 32bit address

Answer 1

Python's PyIntObject uses a C long internally, which is 64-bit on most 64-bit platforms (excluding 64-bit Windows). However, ctypes assigns the converted result to pa->value.i, where value is a union and the i field is a 32-bit int. For the details, see ConvParam in Modules/_ctypes/callproc.c, lines 588-607 and 645-664. ctypes was developed on Windows, where a long is always 32-bit, but I don't know why this hasn't been changed to use the long field instead, i.e. pa->value.l. Probably, it's just more convenient most of the time to default to creating a C int instead of using the full range of the long.

Anyway, this means you can't simply pass a Python int to create a 64-bit pointer. You have to explicitly create a ctypes pointer. You have a number of options for this. If you're not concerned about type safety, the simplest option for a NumPy array is to use its ctypes attribute. This defines the hook _as_parameter_ that lets Python objects set how they're converted in ctypes function calls (see lines 707-719 in the previous link). In this case it creates a void *. For example, you'd call plate like this:

plate.plate(x.ctypes, y.ctypes, N)

However, this doesn't offer any type safety to prevent the function from being called with an array of the wrong type, which will result in either nonsense, bugs, or a segmentation fault. np.ctypeslib.ndpointer solves this problem. This creates a custom type that you can use in setting the argtypes and restype of a ctypes function pointer. This type can verify the array's data type, number of dimensions, shape, and flags. For example:

import numpy as np
import ctypes

c_npfloat32_1 = np.ctypeslib.ndpointer(
    dtype=np.float32, 
    ndim=1, 
    flags=['C', 'W'])

plate = ctypes.CDLL('libplate.so')

plate.plate.argtypes = [
    c_npfloat32_1,
    c_npfloat32_1,
    ctypes.c_int,
]

N = 3
x = np.ones(N, dtype=np.float32)
y = np.ones(N, dtype=np.float32)

plate.plate(x, y, N)  # the parameter is the array itself