compression | 易学教程

Decompressing a .bz2 file in Python

阅读更多关于 Decompressing a .bz2 file in Python

So, this is a seemingly simple question, but I'm apparently very very dull. I have a little script that downloads all the .bz2 files from a webpage, but for some reason the decompressing of that file is giving me a MAJOR headache. I'm quite a Python newbie, so the answer is probably quite obvious, please help me. In this bit of the script, I already have the file, and I just want to read it out to a variable, then decompress that? Is that right? I've tried all sorts of way to do this, I usually get "ValueError: couldn't find end of stream" error on the last line in this snippet. I've tried to

Best compression technique for binary data? [closed]

阅读更多关于 Best compression technique for binary data? [closed]

I have a large binary file that represents the alpha channel for each pixel in an image - 0 for transparent, 1 for anything else. This binary data needs to be dynamically loaded from a text file, and it would be useful to get the maximum possible compression in it. De-compression times aren't majorly important (unless we're talking a jump of say a minute to an hour), but the files need to be as small as possible. Methods we've tried so far are using run length encoding, then a huffman coding, then converting the binary data to base64, and run length encoding but differentiating between zero

How to compress a Byte array without stream or system io

阅读更多关于 How to compress a Byte array without stream or system io

问题 I'm trying to encode an image into a byte array and send it to a server. the encoding and sending parts wok fine but my problem is that the byte array is too large and takes too long to send so I thought compressing it would make it go faster. but the actual problem is that I CAN NOT use system.io or streams. and I'm targeting .net 2.0. Thank you. 回答1: using System.IO; using System.IO.Compression; code: public static byte[] Compress(byte[] data) { MemoryStream output = new MemoryStream();

Buffered Background InputStream Implementations

阅读更多关于 Buffered Background InputStream Implementations

I've written background InputStream (and OutputStream ) implementations that wrap other streams, and read ahead on a background thread, primarily allowing for decompression/compression to happen in different threads from the processing of the decompressed stream. It's a fairly standard producer/consumer model. This seems like an easy way to make good use of multi-core CPUs with simple processes that read, process, and write data, allowing for more efficient use of both CPU and disk resources. Perhaps 'efficient' isn't the best word, but it provides higher utilisation, and of more interest to

Does GZIP Compression Level Have Any Impact On Decompression

阅读更多关于 Does GZIP Compression Level Have Any Impact On Decompression

问题 I understand that GZIP is a combination of LZ77 and Huffman coding and can be configured with a level between 1-9 where 1 indicates the fastest compression (less compression) and 9 indicates the slowest compression method (best compression). My question is, does the choice of level only impact the compression process or is there an additional cost also incurred in decompression depending on the level used to compress? I ask because typically many web servers will GZIP responses on the fly if

Compressed Json Javascript [closed]

阅读更多关于 Compressed Json Javascript [closed]

问题 Closed. This question is off-topic. It is not currently accepting answers. Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed 3 years ago . Currently I send JSON from an Ajax post to the server which is then converted to objects using the Jackson Mapper. The format is like this {"id":"780710","folderID":"42024","displayOrder":2},{"id":"780724","folderID":"42024","displayOrder":3} What is the best JavaScript library to compress this data and will the

Deleting files after adding to tar archive

阅读更多关于 Deleting files after adding to tar archive

问题 Can GNU tar add many files to an archive, deleting each one as it is added? This is useful when there is not enough disk space to hold both the entire tar archive and the original files - and therefore it is not possible to simply manually delete the files after creating an archive in the usual way. 回答1: With GNU tar, use the option --remove-files . 回答2: I had a task - archive files and then remove into OS installed "tar" without GNU-options. Method: Use "xargs" Suppose, we are have a

Best way to compress string in PHP [duplicate]

阅读更多关于 Best way to compress string in PHP [duplicate]

This question already has answers here : Which compression method to use in PHP? (4 answers) I am compressing the array with gzcompress(json_encode($arr),9). So I am converting array into string with json_encode and then compress with gzcompress. But I could not find the much difference in the size of the resulted string. Before compression size is 488 KB and after compression size is 442 KB. Is there any way I can compress the string further? Thanks in advance. How good the compression of your string will be depends on the data you want to compress. If it consists mainly of random data you

Compress Python Object in Memory

阅读更多关于 Compress Python Object in Memory

问题 Most tutorials on compressing a file in Python involve immediately writing that file to disk with no intervening compressed python object. I want to know how to pickle and then compress a python object in memory without ever writing to or reading from disk. 回答1: I use this to save memory in one place: import cPickle import zlib # Compress: compressed = zlib.compress(cPickle.dumps(obj)) # Get it back: obj = cPickle.loads(zlib.decompress(compressed)) If obj has references to a number of small

How to compress small strings

阅读更多关于 How to compress small strings

问题 I have an sqlite database full of huge number of URLs and it's taking huge amount of diskspace, and accessing it causes many disk seeks and is slow. Average URL path length is 97 bytes (host names repeat a lot so I moved them to a foreign-keyed table). Is there any good way of compressing them? Most compression algorithms work well with big documents, not "documents" of less that 100 bytes on average, but even 20% reduction would be very useful. Any compression algorithms that would work?