PyEval_InitThreads in Python 3: How/when to call it? (the saga continues ad nauseam)

Question

Basically there seems to be massive confusion/ambiguity over when exactly PyEval_InitThreads() is supposed to be called, and what accompanying API

Answer 1

I have seen symptoms similar to yours: deadlocks if I only call PyEval_InitThreads(), because my main thread never calls anything from Python again, and segfaults if I unconditionally call something like PyEval_SaveThread(). The symptoms depend on the version of Python and on the situation: I am developing a plug-in that embeds Python for a library that can be loaded as part of a Python extension. The code needs therefore to run independent of whether it is loaded by Python as main.

The following worked for be with both python2.7 and python3.4, and with my library running within Python and outside of Python. In my plug-in init routine, which is executed in the main thread, I run:

  Py_InitializeEx(0);
  if (!PyEval_ThreadsInitialized()) {
    PyEval_InitThreads();
    PyThreadState* mainPyThread = PyEval_SaveThread();
  }

(mainPyThread is actually some static variable, but I don't think that matters as I never need to use it again).

Then I create threads using pthreads, and in each function that needs to access the Python API, I use:

  PyGILState_STATE gstate;
  gstate = PyGILState_Ensure();
  // Python C API calls
  PyGILState_Release(gstate);