Cache large volume of json result on client side

Question

I\'ve a asp.net mvc application which returns the JSON result containing upto n number of years worth of data which then gets rendered on Javascript chart.

In order to h

Answer 1

First of all, where is the database? If you are on a local network with gigabit LAN, then hitting it won't be a problem. However, that is not true over the internet. People have limited bandwidth, especially on mobile, and thus you should limit your HTTP calls. Also, less HTTP calls means less strain on the server.

Here are some tips:

• Consider pagination

  When loading "2 years worth", I imagine a lot, like a 100+ page thesis. Consider paginating data instead of loading them all at once. This saves you bandwidth as well as cache space (If ever it's limited).

  How to: Have the server script slice up the data according to what the client wants. It's pretty easy to create pagination in SQL using LIMIT in the query. The logic is like starting_item = (page_needed - 1) * items_per_page

• JSONify data

  Use JSON for transporting data to and from the network. Aside from being lightweight, it's also structured. It will be easier to parse and store later on.

  How to: PHP has a json_encode function to convert arrays into JSON strings. I assume your framework has a similar feature. Have the string echoed on a page then use JSON.parse to convert from JSON string to a JS object. JSON methods come native in modern browsers but if you need to cater old browsers, Crockford has a library to parse it

• Use a well known storage framework

  If a persistent storage is needed for cache across page, I recently came across PersistJS which abstracts localStorage to ones available on the browser. Also, here's a JS implementation of LZW. Keep it handy since localstorage use strings to store data and it has a 5-10MB limit.

  How to: convert the data into a string using JSON.stringify and store it with PersistJS. Then for retrieval, get the string and parse it back using JSON.parse()

• Call only when needed

  Have the cache system only call the server if something is modified, added or if something isn't there. If the data is there, why should you call the server for it?

• Sync the cache

  If you fear of stale data, then have some AJAX sync your cache system by using some method of live data fetching as described in this wiki about Comet.

The last two points depend on your cache framework. But BackboneJS allows it's models and collections to sync to the server, which have the same functionality I mentioned.