Change output format for MySQL command line results to CSV

Question

I want to get headerless CSV data from the output of a query to MySQL on the command line. I\'m running this query on a different machine from the MySQL server, so all those

Answer 1

It is how to save results to CSV on the client-side without additional non-standard tools. This example uses only mysql client and awk.

One-line:

mysql --skip-column-names --batch -e 'select * from dump3' t | awk -F'\t' '{ sep=""; for(i = 1; i <= NF; i++) { gsub(/\\t/,"\t",$i); gsub(/\\n/,"\n",$i); gsub(/\\\\/,"\\",$i); gsub(/"/,"\"\"",$i); printf sep"\""$i"\""; sep=","; if(i==NF){printf"\n"}}}'

Logical explanation of what is needed to do

1. First, let see how data looks like in RAW mode (with --raw option). the database and table are respectively t and dump3

   You can see the field starting from "new line" (in the first row) is splitted into three lines due to new lines placed in the value.

mysql --skip-column-names --batch --raw -e 'select * from dump3' t

one line        2       new line
quotation marks " backslash \ two quotation marks "" two backslashes \\ two tabs                new line
the end of field

another line    1       another line description without any special chars

2. OUTPUT data in batch mode (without --raw option) - each record changed to the one-line texts by escaping characters like \ and new-lines

mysql --skip-column-names --batch -e 'select * from dump3' t

one line      2  new line\nquotation marks " backslash \\ two quotation marks "" two backslashes \\\\ two tabs\t\tnew line\nthe end of field
another line  1  another line description without any special chars

3. And data output in CSV format

The clue is to save data in CSV format with escaped characters.

The way to do that is to convert special entities which mysql --batch produces (\t as tabs \\ as backshlash and \n as newline) into equivalent bytes for each value (field). Then whole value is escaped by " and enclosed also by ". Btw - using the same characters for escaping and enclosing gently simplifies output and processing, because you don't have two special characters. For this reason all you have to do with values (from csv format perspective) is to change " to "" whithin values. In more common way (with escaping and enclosing respectively \ and ") you would have to first change \ to \\ and then change " into \".

And the commands' explanation step by step:

# we produce one-line output as showed in step 2.
mysql --skip-column-names --batch -e 'select * from dump3' t

# set fields separator to  because mysql produces in that way
| awk -F'\t' 

# this start iterating every line/record from the mysql data - standard behaviour of awk
'{ 

# field separator is empty because we don't print a separator before the first output field
sep=""; 

-- iterating by every field and converting the field to csv proper value
for(i = 1; i <= NF; i++) { 
-- note: \\ two shlashes below mean \ for awk because they're escaped

-- changing \t into byte corresponding to  
    gsub(/\\t/, "\t",$i); 

-- changing \n into byte corresponding to new line
    gsub(/\\n/, "\n",$i); 

-- changing two \\ into one \  
    gsub(/\\\\/,"\\",$i);

-- changing value into CSV proper one literally - change " into ""
    gsub(/"/,   "\"\"",$i); 

-- print output field enclosed by " and adding separator before
    printf sep"\""$i"\"";  

-- separator is set after first field is processed - because earlier we don't need it
    sep=","; 

-- adding new line after the last field processed - so this indicates csv record separator
    if(i==NF) {printf"\n"} 
    }
}'