How is google search by image implemented?

Question

I can just drag and drop any image in google and get results. :)

How is it implimented ? What is the idea behind the algorithm ?

Is that image data converted to

Answer 1

I think that google images uses a 3 combined algorithm

1. detect image's deformation (by rotation, scale, illumination,...)
2. detect by the colors
3. detect by the visual similarity

and a fourth algorithm that is a secret by google (to ranking for example) ;)

(see here -> http://www.quora.com/Algorithms/What-is-the-algorithm-used-by-Google-Search-by-Image-1 )

---

Update 2016

My original answer was on 2012 - in the meanwhile other studies and research have taken more and more importance and I learn some new stuff. ;-)

In my opinion the mains "philosophies" about image detections are three:

1. machine learning algorithms
2. deep learning (a machine learning algorithm)
3. (pattern recognition)

Today, I think that pattern recognition has lost its importance: machine learning is in my opinion the right way to work for searching by image.

With machine learning you can even search for similarly match (for example faces - that obviously are not equals between them). The difficult is how you will to teach correctly your machine. Different approach can be taken.

Deep learning is simply a machine learning algorithm. It goes deeper using differents layers to match a possible image, some example of layers could be:

• pixel-vector (color, intensity,..)
• shapes
• edges
• ...