disk-based | 易学教程

Java disk-based key-value storage

阅读更多关于 Java disk-based key-value storage

问题 Is there an efficient Java implementation of a filesystem-based key-value storage with the following features: Store, overwrite, and retrieve byte arrays by a unique ID (may be assigned by the storage) No memory caching (read means read from file system, write means write to file system immediately) Total data size up to few terabytes Number of stored objects up to hundreds of millions Manageable number of file system objects (to move/copy/delete entire storage on file system level) Will

Java: fast disk-based hash set

阅读更多关于 Java: fast disk-based hash set

问题 I need to store a big hash set, able to contain up to approx 200 millions 40 bit values. Storing it as 200 millions 64 bit value would be acceptable (despite the 200 millions * 16 bits loss). The requirements are: tiny memory footprint (disk space ain't an issue, memory is) fast contains(long l) and add(long l) methods (much faster than SQL) embedded free and without nasty licensing (no Berkeley DB). LGPL fine. no false positive and no false negative, so things like disk-based Bloom Filters

Disk backed dictionary/cache for c#

阅读更多关于 Disk backed dictionary/cache for c#

问题 I'm looking for a drop in solution for caching large-ish amounts of data. related questions but for different languages: Python Disk-Based Dictionary Disk-backed STL container classes? Close question in different terms: Looking for a simple standalone persistant dictionary implementation in C# I don't need (or want to pay anything for) persistence, transactions, thread safety or the like and want something that is not much more complex to use than a List<> or Dictionary<>. If I have to write

Disk backed dictionary/cache for c#

阅读更多关于 Disk backed dictionary/cache for c#

I'm looking for a drop in solution for caching large-ish amounts of data. related questions but for different languages: Python Disk-Based Dictionary Disk-backed STL container classes? Close question in different terms: Looking for a simple standalone persistant dictionary implementation in C# I don't need (or want to pay anything for) persistence, transactions, thread safety or the like and want something that is not much more complex to use than a List<> or Dictionary<>. If I have to write code, I'll just save everything off as files in the temp directory: string Get(int i) { File

Python Disk-Based Dictionary

阅读更多关于 Python Disk-Based Dictionary

I was running some dynamic programming code (trying to brute-force disprove the Collatz conjecture =P) and I was using a dict to store the lengths of the chains I had already computed. Obviously, it ran out of memory at some point. Is there any easy way to use some variant of a dict which will page parts of itself out to disk when it runs out of room? Obviously it will be slower than an in-memory dict, and it will probably end up eating my hard drive space, but this could apply to other problems that are not so futile. I realized that a disk-based dictionary is pretty much a database, so I

Python Disk-Based Dictionary

阅读更多关于 Python Disk-Based Dictionary

问题 I was running some dynamic programming code (trying to brute-force disprove the Collatz conjecture =P) and I was using a dict to store the lengths of the chains I had already computed. Obviously, it ran out of memory at some point. Is there any easy way to use some variant of a dict which will page parts of itself out to disk when it runs out of room? Obviously it will be slower than an in-memory dict, and it will probably end up eating my hard drive space, but this could apply to other