How do you deal with lots of small files?

Question

A product that I am working on collects several thousand readings a day and stores them as 64k binary files on a NTFS partition (Windows XP). After a year in production the

Answer 1

One common trick is to simply create a handful of subdirectories and divvy up the files.

For instance, Doxygen, an automated code documentation program which can produce tons of html pages, has an option for creating a two-level deep directory hierarchy. The files are then evenly distributed across the bottom directories.