Unzipping a gz file in c# : System.IO.InvalidDataException: 'The archive entry was compressed using an unsupported compression method.'

问题

I have followed Microsoft's recommended way to unzip a .gz file :

https://docs.microsoft.com/en-us/dotnet/api/system.io.compression.gzipstream?view=netcore-3.1

I am trying to download and parse files from the CommonCrawl. I can successfully download them, and unzip them with 7-zip

However, in c# I get:

System.IO.InvalidDataException: 'The archive entry was compressed using an unsupported compression method.'

public static void Decompress(FileInfo fileToDecompress)
        {
            using (FileStream originalFileStream = fileToDecompress.OpenRead())
            {
                string currentFileName = fileToDecompress.FullName;
                string newFileName = currentFileName.Remove(currentFileName.Length - fileToDecompress.Extension.Length);

                using (FileStream decompressedFileStream = File.Create(newFileName))
                {
                    using (GZipStream decompressionStream = new GZipStream(originalFileStream, CompressionMode.Decompress))
                    {
                        decompressionStream.CopyTo(decompressedFileStream);
                        Console.WriteLine($"Decompressed: {fileToDecompress.Name}");
                    }
                }
            }
        }

The file is from there:

https://commoncrawl.s3.amazonaws.com/crawl-data/CC-MAIN-2020-16/segments/1585370490497.6/wet/CC-MAIN-20200328074047-20200328104047-00010.warc.wet.gz

Does anyone know what the problem could be? do I need a special library?

回答1:

I am not sure what the issue is but after reading this post

Decompressing using GZipStream returns only the first line

I changed to SharZipLib (http://www.icsharpcode.net/opensource/sharpziplib/) and it worked

回答2:

I took another look at that source file and it appears to be a large number (52,593) of gzip streams concatenated together. Apparently legal according to the spec but it would seem GZipStream doesn't handle that well. Glad you got it working!

来源：https://stackoverflow.com/questions/61446785/unzipping-a-gz-file-in-c-sharp-system-io-invaliddataexception-the-archive-en

标签

gzip

common-crawl