Custom Text filter for DC.js dataTable

Question

I'm building a dashboard to show some data. I have several charts and a table listing all of the data. I'm trying to add search functionality to filter the chart. I have a bunch of companies and some data about each. So if I search for "Appl" only companies that start with "Appl" will be listed in the data table and the charts will reflect this.

The only issue I have with the current implementation is when I change this filter or clear it. The data seems fine, but the charts render incorrectly. They don't return to their original positions when cleared, or they add extra data somehow. Any tips would be appreciated.

 $("#table-search").on('input',function(){
   text_filter(companyDimension,this.value);//companyDimension is the dimension for the data table

function text_filter(dim,q){
 dashTable.filterAll();
 var re = new RegExp(q,"i")
 if (q!='')
 {
    dim.filter(function(d){
        if (d.search(re)==0)
            return d;
    });
}
dc.redrawAll();
graphCustomizations();  }});

dc.js code

var ndx = crossfilter(resource_data);
//Dimensions 
companyDimension = ndx.dimension(function(d){
    return d["Company Name"]
});
dashTable.width(800).height(800)
    .dimension(companyDimension)
    .group(function(d){
        return "List of all Selected Companies";
    })
    .size(1774)
    .columns([
            function(d){return d["Company Name"]; },
            function(d){return d["Revenue Source"];},
            function(d){return d["Commodity"];},
            function(d){return "$"+parseFloat(d["Revenue"]).formatMoney(0,'.',',');}
        ])
    .sortBy(function(d){return d["Company Name"]})
    .order(d3.ascending);

That's about it, the charts are just filtering with different dimensions on the same crossfilter object.

I've tried doing several things to the text_filter function such as, dim.filterAll(), dim.filter(null), dc.renderAll(). When I inspect the data in the dimension, it is correct before and after each filter, the other charts just don't seem to be handling it correctly.

I've tried adding a basic filter to the dc dataTable directly, but I can't get it to work with a custom filter function. So I can do something like dashTable.filter(q) and it will work, but I have to give it the entire company name for it to display anything, but the charts render correctly when I apply it and remove it. I've tried using dashTable.filterHandler() but it always returns an error, but if you know how to get that to work, I would be curious, because I couldn't get it to function even with the example in dc.js's documentation.

Any help would be greatly appreciated.

EDIT:

Here's a fiddle of the mostly complete code, I jumbled some code together to get it working. http://jsfiddle.net/rbristow/HW52d/1/

To reproduce the bug, enter a letter in the search box then clear it and enter another letter, you can see the total not resetting correctly.

Answer 1

In this block:

if (q != '') {
    dim.filter(function(d) {
        if (d.search(re) == 0)
            return d;
    });
}

Your filter needs to be:

dim.filter(function(d) { return 0 == d.search(re); });

But then, you're not applying any filter to dim if q == '' so it should be

if (q != '') {
    dim.filter(function(d) {
        return 0 == d.search(re);
    });
} else {
    dim.filterAll();
}

Explanation:

In crossfilter.js the return value of your filter callback is tested like this:

if (!(filters[k = index[i]] & one) ^ (x = f(values[i], i))) {
    if (x) filters[k] &= zero, added.push(k);
    else filters[k] |= one, removed.push(k);
}

If the filter returns true and the item is already in the current view, it's not supposed to do anything. true ^ true -> false.

But in your case, true is being xor-ed with a string -- note, this is bitwise xor, not logical, as Javascript lacks a logical xor -- which will always evaluate to a true value. So the values you want in your filtered set are being put into added when they should be left alone.

It's an oddball use of a bitwise xor. I looked this up on SO and the top voted answer to Why is there no logical xor in JavaScript? contains "Bitwise XOR is extremely useful, but in all my years of programming I have never needed a logical XOR." Given that crossfilter.js emphasizes performance maybe they drop some error checks and want to use fast "mathy" operations.