Haskell: Can I perform several folds over the same lazy list without keeping list in memory?

Question

My context is bioinformatics, next-generation sequencing in particular, but the problem is generic; so I will use a log file as an example.

The file is very large (G

Answer 1

To process lazy data muiltiple times, in constant space, you can do three things:

• re-build the lazy list from scratch n times
• fuse n passes into a single sequential fold that does each step, in lock step.
• use par to do n parallel traversals at the same time

Those are your options. The last one is the coolest :)