How do you specify filenames within a zip when creating it on the command line from a pipe?

Question

I'm trying to create a zip file from file contents which are being piped in, e.g.

mysql [params and query] | zip -q output.zip -

This writes the zip correctly, but when you open the zip, the file within it is called "-". Is there any way of specifying what the filename of the piped in data should be within the zip?

Answer 1

From what i can gather you cannot do both with the zip command, i mean you cannot both specify the filenames and pipe the content. You can either pipe the contents and the resulting file is - or you can pipe the filenames with -@.

That does not mean that doing so is impossible using other techniques. I outline one of those below. It does mean that you have to have PHP installed and the zip extension loaded.

There could be a whole bunch of other ways to do it. But this is the easiest that I know of. Oh and it's a one-liner too.

This is a working example using PHP

echo contents | php -r '$z = new ZipArchive();$z->open($argv[1],ZipArchive::CREATE);$z->addFromString($argv[2],file_get_contents("php://stdin"));$z->close();' test.zip testfile

To run on windows just swap single and double quotes. Or just place the script in a file.

"test.zip" is the resulting Zip file, "testfile" is the name of the contents that are being piped into the command.

Result from unzip -l test.zip

Archive:  test.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        6  01-07-2010 12:56   testfile
---------                     -------
        6                     1 file

And here is a working example using python

echo contents | python -c "import sys
import zipfile
z = zipfile.ZipFile(sys.argv[1],'w')
z.writestr(sys.argv[2],sys.stdin.read())
z.close()
" test5.zip testfile2

Result of unzip -l

Archive:  test5.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        9  01-07-10 13:43   testfile2
 --------                   -------
        9                   1 file

Result of unzip -p

contents

Answer 2

You can use a named pipe, and send the request output to it, while zipping from it.

mkfifo output.txt ; mysql [params and query] > output.txt & zip output.zip -FI output.txt ; rm output.txt

Answer 3

You can do this.

ls | zip test.zip -@

this is done from the notion that i have 3 files in the dir.

-rw-rw-r-- 1 xxx domain users   6 Jan  7 11:41 test1.txt
-rw-rw-r-- 1 xxx domain users   6 Jan  7 11:41 test2.txt
-rw-rw-r-- 1 xxx domain users   6 Jan  7 11:41 test3.txt

and the file itself, the result is then

Archive:  test.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        6  01-07-10 11:41   test1.txt
        6  01-07-10 11:41   test2.txt
        6  01-07-10 11:41   test3.txt
 --------                   -------
       18                   3 files

From the Linux Zip Man page

If the file list is specified as -@, zip takes the list of input files from standard input.

Answer 4

I couldn't manage with the PHP answer (out of memory on bigger mysql dumps), and the FIFO was not working as I wanted, so my solution is to rename the file inside the ZIP archive after running the dump, using zipnote (which is included with the zip package on Debian).

mysql [params and query] | zip -q output.zip -
echo -e "@ -\n@=newname.sql" | zipnote -w output.zip

Answer 5

mysql [params and query] | python3 -c "import sys as s,os,zipfile as m;z=m.ZipFile(os.fdopen(s.stdout.fileno(),'wb'),'w');z.writestr(s.argv[1],s.stdin.read());z.close()" 'filename.sql' > output.zip

Python 3.5 added support for writing to unseekable streams.

It's not pretty but works.

Answer 6

You can try:

mysql [params and query] | zip -q output.zip -@