Maximum Data File Size for D3.js

Question

What is the limiting factor with data file size for D3js? Is it just how long it takes for the file to load from the server to the client?

I'm trying to create a road map of Chicago, where you can hover over the road to get it's name and highlight it. The file size was 125 MB from the city. I stripped it down to 30 MB by removing unnecessary information, and then converted it to a TopoJSON file bringing it down to 5.1MB. Is this an unruly file size?

Answer 1

There are three limiting factors when working with large amounts of data:

• The size of the file to load and the connection speed of the server/client.
• The processing to be done on the data.
• The rendering of the visual elements representing the data.

Without more details on your application, it's hard to guess which one it is, but in my experience it is almost always the third. The rendering of the elements usually takes most of the time it takes to display a visualisation.

The debug/web development tools in your browser should give you a better idea of where the bottleneck is; in particular you should be able to find out quite easily how long it takes to load the JSON. If that is significantly shorter than the time to display the visualisation, the problem is almost certainly the rendering of the elements.

In this case, there are two main things you can do. You can either reduce the number of elements shown (e.g. by aggregating), or switch to a different rendering mechanism. In particular, canvas renders significantly faster than SVG. If you make that switch, you lose the interactivity that SVG makes quite easy, so it's a tradeoff that depends on your particular application.