D3 making new, smaller CSV file

问题

I'm stuck with a quite simple problem and need help.

I have a big CSV file with 50 columns which i absolutely can't modifie.

Now i want to make a chart where i only need 5-6 columns out of it.

My idea was now to make a new "data2" which contains only these 5-6 columns (with key and evertything) and work with this data2.

But i'm not able to create this data2.

To filter which columns i need i wanted to work with regex. Something like this:

d3.keys(data[0]).filter(function(d) { return d.match(/.../); })

But how do i create the new data2 then? I'm sure i need to work with d3.map but even with the api i'm not able to understand how it works correctly.

Can someone help me out?

回答1:

Firstly, your question's title is misleading: you're not asking about making a smaller CSV file, since the file itself is not changed. You're asking about changing the data array created by D3 when that CSV was parsed.

That brings us to the second point: you don't need to do that. Since you already lost some time/resources loading the CSV and parsing that CSV, the best idea is just keeping it the way it is, and using only those 5 columns you want. If you try to filter some columns out (which means deleting some properties from each object in the array) you will only add more unnecessary tasks for the browser to execute. A way better idea is changing the CSV itself.

However, if you really want to do this, you can use the array property that d3.csv creates when it loads an CSV, called columns, and a for...in loop to delete some properties from each object.

For instance, here...

var myColumns = data.columns.splice(0, 4);

... I'm getting the first 4 columns in the CSV. Then, I use this array to delete, in each object, the properties regarding all other columns:

var filteredData = data.map(function(d) {
  for (var key in d) {
    if (myColumns.indexOf(key) === -1) delete d[key];
  }
  return d;
})

Here is a demo. I'm using a <pre> element because I cannot use a real CSV in the Stack snippet. My "CSV" has 12 columns, but my filtered array keeps only the first 4:

var data = d3.csvParse(d3.select("#csv").text());
var myColumns = data.columns.splice(0, 4);
var filteredData = data.map(function(d) {
  for (var key in d) {
    if (myColumns.indexOf(key) === -1) delete d[key];
  }
  return d;
})

console.log(filteredData)

pre {
  display: none;
}

<script src="https://d3js.org/d3.v4.min.js"></script>
<pre id="csv">foo,bar,baz,foofoo,foobar,foobaz,barfoo,barbar,barbaz,bazfoo,bazbar,bazbaz
1,2,5,4,3,5,6,5,7,3,4,3
3,4,2,8,7,6,5,6,4,3,5,4
8,7,9,6,5,6,4,3,4,2,9,8</pre>

来源：https://stackoverflow.com/questions/44858992/d3-making-new-smaller-csv-file

标签

javascript

csv

dictionary

d3.js

filter