cpython

Given a file /myfiles/file_with_invalid_encoding.txt with invalid UTF8 as: parse this correctly Føö»BÃ¥r also parse this correctly I am using the builtin Python open function from the C API as follows the minimal example (excluding C Python setup boilerplate): const char* filepath = "/myfiles/file_with_invalid_encoding.txt"; PyObject* iomodule = PyImport_ImportModule( "builtins" ); if( iomodule == NULL ) { PyErr_PrintEx(100); return; } PyObject* openfunction = PyObject_GetAttrString( iomodule, "open" ); if( openfunction == NULL ) { PyErr_PrintEx(100); return; } PyObject* openfile = PyObject

I'm tyring to compile Yenc for Python 3.2. I noticed that gcc complained about a non-declared function PyString_Type , so I replaced it with its replacement PyBytes_Type as according to the documentation . However, gcc also complained about an undeclared function called PyFile_Type . I googled a bit and found: Python 3.x replaces the PyFile_Type extension type with an abstract interface and specific implementation types. Unfortunately it doesn't make any of this directly available with a C level api. source I am by no means a C-programmer, which leaves me unable to solve this issue. What

I am trying to learn the intern mechanism of python using in the implementation of string object. But in both PyObject *PyString_FromString(const char *str) and PyObject *PyString_FromStringAndSize(const char *str, Py_ssize_t size) python interned strings only when its size is 0 or 1. PyObject * PyString_FromString(const char *str) { fprintf(stdout, "creating %s\n", str);------------[1] //... //creating... /* share short strings */ if (size == 0) { PyObject *t = (PyObject *)op; PyString_InternInPlace(&t); op = (PyStringObject *)t; nullstring = op; Py_INCREF(op); } else if (size == 1) {

I'm working on a C extension and am at the point where I want to track down memory leaks. From reading Python's documentation it's hard to understand when to increment / decrement reference count of Python objects. Also, after couple days spending trying to embed Python interpreter (in order to compile the extension as a standalone program), I had to give up this endeavor. So, tools like Valgrind are helpless here. So far, by trial and error I learned that, for example, Py_DECREF(Py_None) is a bad thing... but is this true of any constant? I don't know. My major confusions so far can be listed

What do we call "stack" in Python? Is it the C stack of CPython? I read that Python stackframes are allocated in a heap. But I thought the goal of a stack was... to stack stackframes. What does the stack do then? Oversimplifying slightly: In CPython, when PyEval_EvalFrameEx is evaluating a Python stack frame's code, and comes to a direct function call, it allocates a new Python stack frame, links it up… and then recursively calls PyEval_EvalFrameEx on that new frame. So, the C stack is a stack of recursive calls of the interpreter loop. The Python stack is a stack of Python frame objects,

I was talking with a friend, comparing languages, and he mentioned that Java's automated memory management is superior to Python's as Java's does compaction, while Python's does not - and hence for long-running servers, Python is a poor choice. Without getting into which is better or worse, is his claim true - does CPython's garbage collector not compact memory and, thus, long-running Python processes get more and more fragmented over time? I know that running CPython's garbage collector is optional. Mostly it uses automated reference counting for automated memory management, and as soon as a

I cannot figure how to get the virtual address of the standard mmap objects in Python (from the mmap module). The documented methods only seem to access the memory as array of bytes or as character strings. But I need to access the mmap'ped memory by precisely 2 or 4 bytes at once - because this memory in my application is mapped to hardware registers (think /dev/mem or GPIO or such). Accessing memory in this way is possible with ctypes module - but for this I need the pointer - or virtual address - of the mapping. Currently I overcome this by using the native open() and mmap() functions from

I'm trying to compile my hello.pyx file to an exe using Cython. First step was to compile the hello.pyx into a hello.cpp file using command "cython --cplus --embed hello.pyx". Embed option means to Generate a main() function that embeds the Python interpreter . I'm trying to create an independent exe with no dependencies. In hello.cpp I have an #include "Python.h" , so I'm downloading Python sources from here: http://www.python.org/download/releases/2.6.6/ , choosing Gzipped source tar ball (2.6.6) . I add include dir and get error about missing Python26.lib. So I am trying to compile it. The

Say I have a running CPython session, Is there a way to run the data ( bytes ) from a pyc file directly? (without having the data on-disk necessarily, and without having to write a temporary pyc file) Example script to show a simple use-case: if foo: data = read_data_from_somewhere() else: data = open("bar.pyc", 'rb').read() assert(type(data) is bytes) code = bytes_to_code(data) # call a method from the loaded code code.call_function() Exact use isn't important, but generating code dynamically and copying over a network to execute is one use-case (for the purpose of thinking about this