erlang gen_server with a large state

╄→гoц情女王★ 提交于 2020-01-02 04:39:08

问题


I have a trie (implemented with tuples and lists) with several thousand entries and I would like to support concurrent reads. The memory footprint of the data is in the 10-20 MB range. The trie is built once and read only after that.

What is the recommended way to maintain the state and give clients concurrent access?

Here is what I have tried:

1) Created a gen_server with the trie as the state. This worked fine but, obviously, all calls were serialized.

2) Modified (1) to spawn a new process for each call which takes the state, the request, and From. Each new process traversed the trie and called gen_server:reply/2 with the result. This solution didn't seem to work because memory and CPU usage exploded . I assume this happened because the state was copied to the spawned process for every call.


回答1:


mochiglobal from mochiweb is designed for exactly this kind of use case. Basically it will take your data structure and compile it into a module, so the data is shared (zero copy for module constants). Only works well on data structures that don't change often, but it sounds like that's what you have.

  • https://github.com/mochi/mochiweb/blob/master/src/mochiglobal.erl
  • http://www.erlang.org/pipermail/erlang-questions/2009-March/042503.html



回答2:


if your state is changing often implement your model/structure over ETS.

You can create ETS table with concurrent read/write options which would increase performance.




回答3:


Another approach would be to make a pool of gen_servers (with supervision), and then allocate incoming connections to a server in the pool. This eases the gen_server bottleneck associated with your first approach. This approach also allows some tuning by adjusting the number of processes in the pool. LearnYouSomeErlang has a chapter on this.



来源:https://stackoverflow.com/questions/5130382/erlang-gen-server-with-a-large-state

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!