Multithreading with Python and C api

Question

I have a C++ program that uses the C api to use a Python library of mine. Both the Python library AND the C++ code are multithreaded.

In particular, one th

Answer 1

A correct order of steps to perform what you are trying to do is:

• In the main thread:

  1. Initialize Python using Py_Initialize*.
  2. Initialize Python threading support using PyEval_InitThreads().
  3. Start the C++ thread.

At this point, the main thread still holds the GIL.

• In a C++ thread:
  1. Acquire the GIL using PyGILState_Ensure().
  2. Create a new Python thread object and start it.
  3. Release the GIL using PyGILState_Release().
  4. Sleep, do something useful or exit the thread.

Because the main thread holds the GIL, this thread will be waiting to acquire the GIL. If the main thread calls the Python API it may release the GIL from time to time allowing the Python thread to execute for a little while.

• Back in the main thread:
  1. Release the GIL, enabling threads to run using PyEval_SaveThread()
  2. Before attempting to use other Python calls, reacquire the GIL using PyEval_RestoreThread()

I suspect that you are missing the last step - releasing the GIL in the main thread, allowing the Python thread to execute.

I have a small but complete example that does exactly that at this link.