compression

I have arrays of 1024 bytes (8192 bits) which are mostly zero. Between 0.01% and 10% of bits will be set (random, no pattern). How could these be compressed, given the lack of structure and the relatively small size? (My first thought was to store the distances between set bits. I need 13 bits for each distance, but at worst case 10% occupancy this needs 13 * 816 / 8 = 1326 bytes, which is not an improvement.) This is for ultra-low bandwidth comms, so every byte matters. I've dealt deeply with a similar problem, but my sets are much bigger (30 million possible values with between 1 and 30

I'm trying to figure out how to properly use webpack-html-plugin with the compression plugin , the documentation of latter is a bit scarce. My webpack configuration declares : output: { filename: 'js/[name]-[hash].js', The compression plugin is run at last new CompressionPlugin({ asset: "[path].gz[query]", algorithm: "gzip" }) In the end the scripts are correctly produced and compressed. js/app-caf3b4b3.js.gz 382 kB [emitted] [big] I can declare the preloading of the gzipped file in the index.html template <link rel="preload" href="<%= htmlWebpackPlugin.files.chunks[chunk].entry %>.gz" as=

I want to compress/decompress and serialize/deserialize String content. I'm using the following two static functions. /** * Compress data based on the {@link Deflater}. * * @param pToCompress * input byte-array * @return compressed byte-array * @throws NullPointerException * if {@code pToCompress} is {@code null} */ public static byte[] compress(@Nonnull final byte[] pToCompress) { checkNotNull(pToCompress); // Compressed result. byte[] compressed = new byte[] {}; // Create the compressor. final Deflater compressor = new Deflater(); compressor.setLevel(Deflater.BEST_SPEED); // Give the

Is there a way to know if the byte[] has been compressed (or not) by GzipStream .net class? EDIT: Just want to know if the byte[] array has been compressed (since I will always be using GzipStream to compress and decompress) A GZipStream is a DeflateStream with an additional header and trailer. The format is specified in RFC 1952 . The .NET 4.0 GZipStream class writes the following bytes as header: byte[] headerBytes = new byte[] { 0x1f, 0x8b, 8, 0, 0, 0, 0, 0, 4, 0 }; if (compressionLevel == 10) { headerBytes[8] = 2; } The trailer consists of a CRC32 checksum and the length of the

Im producing some automated tasks at work where i need to zip certain files and/or folders. What im trying to do is getting zip the text files in folder 1 which contains 4 txt files. Executing this command gives an error but still zips the txt files : Exception calling "CreateFromDirectory" with "4" argument(s): "The directory name is invalid. " At line:15 char:13 + [System.IO.Compression.ZipFile]::CreateFromDirectory($Source, "$Sour ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException What i got

I have an opportunity to preset dictionary for deflate compression. It makes sense in my case, because data to be compressed is relatively small 1kb-3kb and I have a large sample of representative examples. Data to be compressed consists of arbitrary sequence of bytes, so tokenization etc. is not a good way to go. Also, data shows a lot of repetition (between data examples), so good dictionary could potentially give very good results. The question is how calculate good dictionary? Is there an algorithm which calculates optimal dictionary (given sample data)? I started looking at prefix trees,

I'm working on a project where I edit the pixels of a jpg in PHP using the GD library. Its very crucial and key to my project that the output from PHP retains the pixel values I set (cough-steganography-cough). This is the first time I've attempted image manipulation in PHP. I've had no problems before in my Java implementations, so I was foolish not to investigate GD's jpeg compression quality before pursuing further. It turns out that after all my effort, my code does not function the way its supposed to. I am pretty sure it is not my code (tested the encode and decode functions on edited

I'm looking for a JavaScript implementation of a string compress/decompress algorithm where data is created at the client side and stored in hidden fields within HTML forms. I read about gzip , but it compresses the data server side whereas in my case I want to compress it client side, send it to the server, or receive it from server to decompress it again at client side. I found this LZF Compression example based on LZFjs but it will generate binary data which needs to be processed and stored in a hidden form field, and it works on files rather than pure data. Suggestions on a pure data

I'm looking for a way to shorten an already short string as much as possible. The string is a hostname:port combo and could look like " my-domain.se:2121 " or " 123.211.80.4:2122 ". I know regular compression is pretty much out of the question on strings this short due to the overhead needed and the lack of repetition but I have an idea of how to do it. Because the alphabet is limited to 39 characters ( [a-z][0-9]-:. ) every character could fit in 6 bits. This reduce the length with up to 25% compared to ASCII. So my suggestion is somthing along these lines: Encode the string to a byte array