Robust and fast checksum algorithm?

Question

Which checksum algorithm can you recommend in the following use case?

I want to generate checksums of small JPEG files (~8 kB each) to check if the content changed.

Answer 1

• CRC-32 comes into mind mainly because it's cheap to calculate

• Any kind of I/O comes into mind mainly because this will be the limiting factor for such an undertaking ;)

• The problem is not calculating the checksums, the problem is to get the images into memory to calculate the checksum.

• I would suggest "stagged" monitoring:

  • stage 1: check for changes of file timestamps and if you detect a change there hand over to...
    (not needed in your case as described in the edited version)

  • stage 2: get the image into memory and calculate the checksum

• For sure important as well: multi-threading: setting up a pipeline which enables processing of several images in parallel if several CPU cores are available.