问题
Possible Duplicate:
How to refer to children in a tree with millions of nodes
I'm trying to implement a tree which will hold millions of nodes, which in turn can have an unspecified number of children nodes.
To achieve this (since each node can have more than one child node), I'm storing a node's children within a Dictionary data structure. As a result of this, when each object node is created (out of millions), I've got a node object which contains a character value stored in the respective node, aswell as a separate Dictionary structure which holds a reference to the children nodes.
My tree works for a few thousand nodes, however when it reaches millions of nodes, an out of memory exception occurs. Is this due to the fact that each one of the millions of nodes running in memory also has its own Dictionary? i.e. I've got millions of objects running?
I need to have these objects running in memory, and cannot use files or databases. Could anyone suggest a solution?
回答1:
Your OOM exception may be due to LOH fragmentation, rather than actually running out of memory. You could try switching to SortedDictionary, which uses a red-black tree, rather than Dictionary, which uses a hashtable, and see if that improves matters. Or you could implement your own tree structure.
回答2:
You could try to use Windows with 64 bits and compile the program at 64 bits. This will give you much more memory... It isn't a magic bullet (there are still limits to how much big a memory structure can be)
回答3:
You can read about Memory Limits for Windows Releases.
http://msdn.microsoft.com/en-us/library/aa366778(v=vs.85).aspx
Think about using 64-bit..
Take a look on this question: Is there a memory limit for a single .NET process
For your scenario try using BigArray
http://blogs.msdn.com/b/joshwil/archive/2005/08/10/450202.aspx
回答4:
The solution isn't going to be easy.
Options are:
Offload some of those nodes to disk, where you have a greater amount of "workspace" to deal with. Only load in memory what really really needs to be there. Due to the radical difference between disk and RAM speeds this could result in a huge performance penalty.
Increase the amount of RAM in the machine to accommodate what you are doing. This might necessitate a move to 64 bit (if it is currently a 32 bit app). Depending on your real memory requirements this could be pretty expensive. Of course, if you have a 32bit app now AND have plenty of RAM available, switching to 64 bit is going to at least get you above the 3 to 4GB range...
Stream line each node to have a much much smaller footprint. In other words, do you need everything Dictionary offers or can you do with just a struct defining the left and right links to other nodes? Basically, take a look at how you need to process this and look at the traditional ways of dealing with tree data structures.
回答5:
Another solution I can suggest if your only restriction is no file access (if you're opposed to ANY sort of database, this answer is moot) - is to use an in-memory database (like SQlite) or something else that provides similar functionality. This will help you in many ways, not the least of which is that the database will perform memory management for you and there are many proven algorithms for storing large trees in databases that you can adapt.
Hope this helps!
来源：https://stackoverflow.com/questions/9670080/memory-exception-in-c-sharp