psycopg2 copy_expert() - how to copy in a gzipped csv file?

Question

If my table is schema_one.table_five and my file name is file_to_import.csv.gz, what args do I give the copy_expert() cmd in order to copy the file contents into the table?<

Answer 1

The file argument to copy_expert should be a file like object, not the file name. For a regular csv file you could use:

with open("file_to_import.csv",  'rb') as this_file:
    cur.copy_expert(this_copy, this_file)

For a gzipped file you could use the gzip module to open the file:

import gzip
with gzip.open("file_to_import.csv.gz",  'rb') as this_file:
    cur.copy_expert(this_copy, this_file)

To change the separator, you'll have to change the COPY statement. See the COPY docs for more information. It might be easier to use copy_from (which has a optional sep argument) instead of copy_expert.

with gzip.open("file_to_import.csv.gz",  'rb') as this_file:
    cur.copy_from(this_file, 'staging.tbl_testcopy_tmp', sep='|')

There isn't a command to automatically import all the files in the directory, you'll have to get a listing of the directory's contents and loop through it.