compression

Noticing that byte-pair encoding (BPE) is sorely lacking from the large text compression benchmark, I very quickly made a trivial literal implementation of it. The compression ratio - considering that there is no further processing, e.g. no Huffman or arithmetic encoding - is surprisingly good. The runtime of my trivial implementation was less than stellar, however. How can this be optimized? Is it possible to do it in a single pass? This is a summary of my progress so far: Googling found this little report that links to the original code and cites the source: Philip Gage, titled 'A New

I want to summarize rather than compress in a similar manner to run length encoding but in a nested sense. For instance, I want : ABCBCABCBCDEEF to become: (2A(2BC))D(2E)F I am not concerned that an option is picked between two identical possible nestings E.g. ABBABBABBABA could be (3ABB)ABA or A(3BBA)BA which are of the same compressed length, despite having different structures. However I do want the choice to be MOST greedy. For instance: ABCDABCDCDCDCD would pick (2ABCD)(3CD) - of length six in original symbols which is less than ABCDAB(4CD) which is length 8 in original symbols. In terms

I would like to decompress the files in different directories which are in different routes. And codes as below and the error is invalid data stream. Please help me out. Thank you so much. import sys import os import bz2 from bz2 import decompress path = "Dir" for(dirpath,dirnames,files)in os.walk(path): for file in files: filepath = os.path.join(dirpath,filename) newfile = bz2.decompress(file) newfilepath = os.path.join(dirpath,newfile) bz2.compress/decompress work with binary data: >>> import bz2 >>> compressed = bz2.compress(b'test_string') >>> compressed b'BZh91AY&SYJ|i\x05\x00\x00\x04\x83

I'm going to store large amount of data (logs) in fragmented PostgreSQL tables (table per day). I would like to compress some of them to save some space on my discs, but I don't want to lose the ability to query them in the usual manner. Does PostgreSQL support such a transparent compression and where can I read about it in more detail? I think there should be some well-known magic name for such a feature. Magnus Hagander Yes, PostgreSQL will do this automatically for you when they go above a certain size. Compression is applied at each individual data value though - not at the full table

For now I'm stuck with IIS6 for ASP.NET-MVC (as in I cant upgrade to Server 2008 yet). It doesnt seem to know that my RESTful URLS are dynamic files and isn't compressing them. All my old .aspx files are compressed (as seen in Fiddler), but not the '/products/1001' type URLS. Is there any way to get IIS6 to compress my ActionResults in IIS6 without using something like an ActionFilter for compression . I'm assuming IIS7 is clever enough to know they're dynamic right. Bonus points if you can tell me how IIS6 even knows which files are dynamic in the first place! As HTTP compression for ASP.NET

Do you know of an online CSS compressor that helps remove redudant/ineffecient CSS declarations and replaces it with more optimized CSS? Meaning, I know that a lot of "compressors" exist that simply remove tabs, remove comments, etc. But what I'm looking for is something smart enough to know that: border-top: 1px solid red; border-bottom: 1px solid red; border-right: 1px solid red; border-left: 1px solid red; is the same as the more efficient: border: 1px solid red; UPDATE : The CSS I'm trying to optimize is at the link below http://tinyurl.com/yhy5ln Will this do? It beautifies, minifies,

I often have to write up specs for video conversion for some of the video production houses that my company's clients work with. Unfortunately, I am a programmer first and "video-guy" on the side, so I don't know too much about all the different codecs. I am looking for a good lossless codec that is both cross-platform (Win and Mac) and cross application (Adobe, Apple, etc). huffyuv is definitely the simplest solution and you will find several cross-platform implementations as C libraries for example. It is easily encapsulated in AVI files and readable by the major players. Format definition

I am trying to implement the zlib.h deflate and inflate functions to compress and decompress a char array (not a file). I would like to know if the following syntax is correct ? Am I missing something or defined something incorrectly ? char a[50] = "Hello World!"; char b[50]; char c[50]; // deflate // zlib struct z_stream defstream; defstream.zalloc = Z_NULL; defstream.zfree = Z_NULL; defstream.opaque = Z_NULL; defstream.avail_in = (uInt)sizeof(a); // size of input defstream.next_in = (Bytef *)a; // input char array defstream.avail_out = (uInt)sizeof(b); // size of output defstream.next_out =