How do I efficiently access gzipped xml from javascript?

Question

I need to efficently access a large gzipped xml file from javascript (actually from Greasemonkey). Unfortunately, the server doesn't provide a Content-Encoding header, and Content-Type is "application/x-gzip", so firefox will not (as far as I can tell) automatically inflate it. If there's a way to fake firefox out, that would be ideal. Short of that, I need some way to efficiently do the inflation...what I'm using now takes about 30 seconds to deflate the 1.2Mb gzipped file; I'd like to get it down under 5 seconds.

(The Greasemonkey script I'm working on can't have any other external server dependencies, so proxying and presenting a Content-Encoding header isn't an option.)

What I'm doing now, I've patched together from several places. To receive the binary data unmolested, I'm using the firefox XMLHTTPRequest overrideMimeType extension:

$.ajax(url, {
    dataType:'text',
    beforeSend:function(xhr){
        xhr.overrideMimeType('text/plain; charset=x-user-defined')
    },
    success:function(data){
        var blob='';
        for (i=0; i<data.length; ++i)
            blob += String.fromCharCode(data.charCodeAt(i) & 0xff);
        ...

Then inflating, using a slightly modified and inlined copy of https://github.com/dankogai/js-deflate/blob/master/rawinflate.js (there are several other javascript inflate libraries out there, all as far as I can tell based on an older library http://www.onicos.com/staff/iz/amuse/javascript/expert/inflate.txt). This is the horrifically slow part.

        // blithely assuming the gzip header won't change,
        // strip a fixed number of bytes from the front
        deflated=RawDeflate.inflate(blob.substring(22,blob.length-8));

Then popping it in an innerHTML property to parse it:

        xmlcontainer=$('<div>');
        // remove <?xml...> prolog
        xmlcontainer.html(deflated.substring(45));
        xmldoc=xmldoc.children();

(I know the last bit could be more properly done with DOMParser's parseFromString, but I didn't get that working yet.)

Answer 1

You're just not going to be able to do noticeably better with this configuration**.

JavaScript is too slow to inflate as fast as desired, and you can't call a binary reliably from JS -- unless AJAXing the data to your own server (which can be the local PC).

Your options for improvement would seem to be:

1. Get the browser to automatically inflate the content. If you have already tried using overrideMimeType to set application/x-gzip, you might try using GM_xmlhttpRequest instead (it's a long shot).

2. Convert this from a GM script to a Firefox add-on. As an add-on, you can access binaries, like 7-Zip, and might even have access to the browser's inflate method. You could probably spoof the mimetype more easily too.

---

**I did notice some trivial opportunities to speed up that inflating JS... things like length checks inside for loops. Alas, given the particulars, probably not going to buy more than a second or two.