Requests module crashes python when numpy is loaded and using process

ⅰ亾dé卋堺 提交于 2020-01-05 13:03:18

问题


Strange title I know, but it is exactly what I see. I am trying to run a requests (2.13.0) command from within a forked process (Mac OSX) using the multiprocessing module. I also happen to use numpy in my code (1.15.1) running on python 3.7. Here are my observations (see code below):

1) Without importing numpy: All works fine 2) Once I import numpy: Code crashes on starting of the forked process. Message given is: objc[45539]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. objc[45539]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

3) I could make it work again by calling a requests call from within the main process once before starting the new process (see commented section in code

4) On python 2.7, all seems to work fine in all cases above.

Sample minimal code to reproduce:

from multiprocessing import Process
import requests
import numpy # remove this import and it works fine on 3.7

def _worker():
    full_url = "http://www.google.com"
    result = requests.get(full_url)
    print(result.text)
    return 0

def run():
    p=Process(target=_worker)
    p.start()
    p.join()

# Add these lines and the code works in 3.7 even with numpy imported
#try:
#    requests.get('http://www.google.com')
#except:
#    pass
run()
print('I am done')

来源:https://stackoverflow.com/questions/52014403/requests-module-crashes-python-when-numpy-is-loaded-and-using-process

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!