How to merge objects attributes from reduce to rereduce function in CouchDB

Question

This is my JSON schema:

{"emp_no": ..,
"salary": ..,
"from_date": ..,
"to_date": ..,
"type" : "salaries"}

{"emp_no": ..,
"title": ..,
"from_date": ..,
"to_date" : ..,
"type" : "titles"}

What i wanted to do, is to find the average salary for each active title. Active titles are document with "from_date" attribute set to "9999-01-01"

Here is my Map Function

function(doc) {
if (doc.type == 'salaries') {
        var dateSalaries = null;
        dateSalaries = doc.to_date.split("-");
        if(dateSalaries[0].localeCompare("9999") == 0){
            emit(doc.emp_no, ["salary", doc.salary] );
        }
    } else if (doc.type == 'titles') {
        var dateTitles = null;
        dateTitles = doc.to_date.split("-");
        if(dateTitles[0].localeCompare("9999") == 0){
            emit(doc.emp_no, ["title", doc.title]);
        }
    }
}

Here is the resulting key value pairs emited:

http://i.imgur.com/o1Qxz.png

Now, i want to reduce it into single key-value pair, with the value outputted is set into javascript object like this

{
    "engineer" : 64342,
    "senior engineer" : 123111,
    "staff" : ...,
    "senior staf" : ...,
    .
    .
    .
}

Here's how i planned to do it: First, in reduce step, i'm gonna return object that merge properties from the same emp_no. Then, in reduce step, i'm gonna create a new object that has properties name based on reduced value before.

It's hard to explain, so here is my reduce function:

function(keys, values, rereduce) {
    var i, l, attr, sal, rv = {};
    if (rereduce) {
        for (i = 0, l = values.length; i<l ; ++i) {
            if (values[i].hasOwnProperty('salary')) {
                attr = values[i].title;
                sal = values[i].salary;
                if (rv[attr] instanceof Array) {
                    rv[attr].push(sal);
                } else{
                    rv[attr] = [];
                    rv[attr].push(sal);
                }
            }           
        }

        for (var x in rv) {
            if (rv.hasOwnProperty(x)) {
                var totalSalary = 0;
                for (i = 0, l = values.length; i<l ; i++) {
                    totalSalary += rv[x][i];
                }
                rv[x] = totalSalary / rv[x].length;
            }
        }

    } else {
        for (i = 0, l = values.length; i<l ; i++) {
            switch (values[i][0]) {
                case "title" : rv["title"] = values[i][1]; break;
                case "salary": rv["salary"] = values[i][1]; break;
            }
        }
    }
    return rv;
}

The resulting value here is reduced value, which is what i expected: http://i.imgur.com/SnlOU.png

But, when i set the grouping value to 'none' in futon, it's not what i wanted:

{Senior Engineer: null, Assistant Engineer: null, Technique Leader: null}

Could someone help me to solves this?

Answer 1

You're pushing CouchDB pretty close to its limits here — using a reduce function to perform a join and everything.

Your problem comes from the fact that CouchDB may apply zero, one or more rereduce steps, but your code assumes that exactly one rereduce step will be performed. I suspect the null results you get come from the fact that the final rereduce step is applied to some results that come from a reduce step and some results that come from a rereduce step.

Here's a small diagram. M is a map step, R is a reduce step, RR is a rereduce step.

[X] [X] [X] [X] [X] [X] [X] [X] [X] [X] 
 |   |   |   |   |   |   |   |   |   | 
(M) (M) (M) (M) (M) (M) (M) (M) (M) (M)
 |   |   |   |   |   |   |   |   |   | 
(==R==) (==R==) (==R==) (==R==) (==R==)
   |       |       |       |       | 
  (== R R ==)     (== R R ==)      | 
       |               |           | 
      (====== R R ======)          | 
               |                   | 
              (======== R R ========)
                         |
                         v
                        [X]

With CouchDB reduce views, it is essential that the data output by your reduce step has the same format as the data output by your rereduce steps. In particular, this means that instead of storing averages, you need to store (sum,count) pairs.

Answer 2

It would make your life a lot easier if you could put the title and salary in the same employee document together.

{
"name" : "Joe",
"title" : "Plumber",
"salary" : 60000
}

then you could easily emit(doc.title, doc.salary) with the built-in _stats reduce function and get a view of salary stats for each title.