Identifying 2 same images using Java

Question

I have a problem in my web crawler where I am trying to retrieve images from a particular website. Problem is that often I see images that are exactly same but different in

Answer 1

Depending on how detailed you want to get with it:

• download the image
• as you download it generate a hash for it
• make a directory where the directory name is the hash value (if the directory does not exist)
• if directory contains 2 or more files then compare the file sizes
• if the file sizes are the same then do a byte by byte comparison of the image to the bytes of the images in the file
• if the bytes are unique then you have a new image

Regardless of if you want to do all that or not you need to:

• download the images
• do a byte-by-byte comparison of the images

No need to rely on any special imaging libraries, images are just bytes.