问题
I am looking at possible technology choices for queues (or perhaps streams are a better description) in a JVM-based system.
Some requirements:
- Must be accessible from the JVM / Java.
- Queues must support sizes larger than the JVM heap, possibly bigger than all available RAM. Thus, support for utilizing the disk (or network) for storage is implied.
- Queues do not currently need to be durable past the process lifetime.
- Most uses of the queue will have a single producer and a single consumer. Concurrency for any particular queue is thus not an issue. (Obviously concurrency is important across queues.)
- Queues are ad-hoc and temporary. They pop into existence, are filled, are drained, and go away.
- Small queues should preferably stay in memory, then shift to slower storage based on resource availability. This requirement could be met above the queuing technology.
I am examining several options but am curious what options I am missing?
回答1:
Use one of available JMS implementations. For example ActiveMQ or Qpid from Jakarta.
回答2:
I ran across this FIFO queue with spill to disk which is kind of interesting and has some of the properties I'm looking for:
http://code.google.com/p/ashes-queue/
回答3:
I have considered using Terracotta's BigMemory as a tool for pushing queue data into direct memory and off-heap.
回答4:
How about using Redis as a messaging queue.It supports both in-memory and can be made persistent once data does not fit the RAM.
回答5:
HSQLDB provides an in-process database engine where you can use RAM, the local disk or a network server to store the database. That might float your boat, especially if you want to seamlessly move to a network solution rather than the local disk later on. Transitioning from small to large queues would then involve moving data from one database to another. There are standard ways to do this, but they might be pretty slow.
There more I think about it, the more I think this is not a good match. For what it's worth, the in-memory DB is very fast in my experience.
来源：https://stackoverflow.com/questions/6729208/what-solutions-exist-for-a-jvm-based-queue-that-is-larger-than-heap