Writing data to disk in parallel?

Question

So I am working on a C++/cli image processing library and am trying to optimize my code. Basically, I am passed a System::Drawing::Bitmap of the image, which I then need to write to disk, perform complex analysis on, and return the results of the analysis. I thought that I could write the image to disk in parallel to speed up things (my algorithm does not modify the image). However, I have not worked with threads much, so I wanted to get your input on what the best way to do this would be.

string ProcessImage(System::Drawing::Bitmap ^bmp, System::String^ targetFile)
{
    bmp->Save(targetFile);
    System::Drawing::Bitmap^ bmp8 = BitmapConvertPixelFormat(bmp, 8); //<-- a function I wrote which converts the 32bpp I am passed into an 8bpp one
    string results = Analyze(bmp8); //<--- takes a good bit of time
    return results;
}

Please let me know your thoughts. Thank you in advance!

Answer 1

Writing/reading in parallel to/from a single mechanical disk is not a good idea because the mechanical head needs to spin every time to service an I/O request, so using multiple threads will just bounce it around needlessly and create overhead.

You can try to benchmark a bit, but I'm afraid you'll just have to resort to using a single thread and writing sequentially.

Answer 2

Queueing off the disk writes to another thread seems like a qood idea, but only to one writer thread per disk so that the complex analysis can run on without the slow disk-writes holding it up.

Answer 3

If it's worth it you can invest some time into understanding how to use Parallel HDF5. It can write to a file in parallel.