Simple explanation of MapReduce?

Question

Related to my CouchDB question.

Can anyone explain MapReduce in terms a numbnuts could understand?

Answer 1

MAP and REDUCE are old Lisp functions from a time when man killed the last dinosaurs.

Imagine you have a list of cities with informations about the name, number of people living there and the size of the city:

(defparameter *cities*
  '((a :people 100000 :size 200)
    (b :people 200000 :size 300)
    (c :people 150000 :size 210)))

Now you may want to find the city with the highest population density.

First we create a list of city names and population density using MAP:

(map 'list
     (lambda (city)
         (list (first city)
               (/ (getf (rest city) :people)
                  (getf (rest city) :size))))
     *cities*)

=>   ((A 500) (B 2000/3) (C 5000/7))

Using REDUCE we can now find the city with the largest population density.

(reduce (lambda (a b)
          (if (> (second a) (second b))
             a
             b))
        '((A 500) (B 2000/3) (C 5000/7)))

 =>   (C 5000/7)

Combining both parts we get the following code:

(reduce (lambda (a b)
          (if (> (second a) (second b))
             a
             b))
        (map 'list
             (lambda (city)
                (list (first city)
                   (/ (getf (rest city) :people)
                      (getf (rest city) :size))))
             *cities*))

Let's introduce functions:

(defun density (city)
   (list (first city)
         (/ (getf (rest city) :people)
            (getf (rest city) :size))))

(defun max-density (a b)
   (if (> (second a) (second b))
          a
          b))

Then we can write our MAP REDUCE code as:

(reduce 'max-density
        (map 'list 'density *cities*))

 =>   (C 5000/7)

It calls MAP and REDUCE (evaluation is inside out), so it is called map reduce.