Too many open files when using NodeJS child_processes.spawn to run scripts

戏子无情 提交于 2019-12-24 02:34:27

问题


Scenario:

Using a master script to spawn a variable number of child processes a variable number of times in order to perform load testing against a server.

The master script initially spawns all the children it can (according to its configuration settings) and then as the children processes exit if there are more runs requested by the config then new children are spun up.

What I'm seeing is an immediate failure upon attempting to spin up the 83rd child process. 83?

I'm not doing anything to explicitly close the files opened as part of the child spawning process but presumably that's not the job of the opening code but the child_processes module code?

I'm very curious about the magic number of 82 child processes. This seems to indicate something about either a limitation in node or some combination of node on my system?

Ideally, there's some lack of knowledge I have that this question will answer or someone can suggest an alternative way to launch child processes of scripts that won't suffer this issue?

I'm also interested in learning about the status of the Web Worker API that is coming to NodeJS. Anyone know anything about that?

The Details:

  • NodeJS v0.4.7
  • Mac OS X v10.6.7
  • ulimit -n = 256
  • magic number of spawned children that will run successfully = 82 (meaning that > 82 spawned procs will throw the "too many open files" error)

Thanks for any help.


回答1:


My guess is the system is doing exactly what you are telling it. 82 processes is 3 open files per process. STDIN, STDOUT, STDERR. Bang. You've hit your ulimit just with the standard 3 file descriptors. Run with ulimit -n 512 and I bet you'll be able to run twice as many children.



来源:https://stackoverflow.com/questions/6036093/too-many-open-files-when-using-nodejs-child-processes-spawn-to-run-scripts

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!