Compression algorithm for JSON encoded packets?

Question

What would be the best compression algorithm to use to compress packets before sending them over the wire? The packets are encoded using JSON. Would LZW be a good one for th

Answer 1

Here is a short test on the compressibility of JSON data original: crime-data_geojson.json 72844By (You can get the file here: https://github.com/lsauer/Data-Hub . The file was picked at random but cannot be representative of average JSON data)

except for zip all archiver parameters were set to ultra

* cm/ nanozip: 
  > 4076/72844
  [1] 0.05595519
* gzip:
  > 6611/72844
  [1] 0.09075559
* LZMA / 7zip
  > 5864/72844
  [1] 0.0805008
* Huffman / zip:
  > 7382/72844
  [1] 0.1013398
* ?/Arc:
  > 4739/72844
  [1] 0.06505683

This means that compression is very high and beneficial. JSON data generally has a high entropy. According to wikipedia

The entropy rate of English text is between 1.0 and 1.5 bits per letter,[1] or as low as 0.6 to 1.3 bits per letter, according to estimates by Shannon based on human experiments

The entropy of JSON data is often well above that. (In an experiment with 10 arbitrary JSON files of roughly equal size i calculated 2.36)