Many short-living processes with global process counter

问题

I need to start many short-lived processes to be started by some kind of spawner process, those processes by itselves do some work, save result (or delegate saving to some saver process) and then exit.

And there also should be some global limit of the number of parallel running processes. And i suppose some kind of queue, because when limit is reached, then spawner could continue to spawn new processes while they i guess should suspend and wait for the room to run.

Could that scheme be implemented easily in erlang ? Does it make sense to have pool of workers to gain some speed or just spawn new process every time is okay ? Could supervisor simple_one_for_one help here ?

回答1:

You can spawn process easily, as already pointed out, and your spawning process could use monitor/2 and a counter to limit and maintain your worker process pool. Your spawning process would spawn an initial set of processes, Monitoring the resulting Pids (i.e., monitor(process, Pid)) and decrementing a counter as appropriate. It would then go into a receive waiting on a {'DOWN', MonitorRef, Type, Object, Info} message indicating that a worker had terminated. As in...

loop(MaxProcesses, LiveProcesses) ->
    receive
       {'DOWN', _Ref, process, _Pid, _Info} ->
           case LiveProcesses of
               N when N =< MaxProcesses -> 
                   spawn_another() %and monitor new pid
                   loop(MaxProcess, LiveProcesses);
               N when N > MaxProcess -> 
                   loop(MaxProcesses, LiveProcesses -1)
           end
        end
    after  %optional if you want the spawner to potentially do something else
        Timeout->
           do more work
           loop(MaxProcesses, LiveProcesses)
        end
    end.

回答2:

You can very easily create/spawn processes that don't do any receive, similar to this:

spawn(fun() -> io:format("Do something and exit~n") end).

Instead of that printing, you could just delegate or something. Not sure if creating a pool for something like that have any sense, unless those processes are running forever and just to send them messages that would be delegated to other entities.

HTH

来源：https://stackoverflow.com/questions/8708084/many-short-living-processes-with-global-process-counter

标签

erlang