How to add single quotes around columns using awk

Question

Just wondering how can I add single quotes around fields, so I can import it to mysql without warnings or errors.

I have a csv file with lots of content.

Answer 1

awk and sed are not going to (easily) determine whether the field separator (,) is escaped or not. The csv file format escapes , characters within fields by surrounding the whole field in double quotes (see Section 2.6 of RFC4180).

As I describe in this answer, a more robust method is to use a csv library, rather than parsing as text using regular expressions and the like.

I found Python's library was the best choice because it's:

1. widely available without onerous dependencies, with the exception of Python itself;
2. not particular sensitive to the version of Python you use;
3. lends itself to being embedded within a shell script; and
4. is quite compact (a one-liner will do!).

Based on the question's tags, I suspect these criteria will be appealing to you too.

Thus, try the following:

QUOTE_CSV_PY="import sys; import csv; csv.writer(sys.stdout, quoting=csv.QUOTE_ALL, quotechar=\"'\").writerows(csv.reader(sys.stdin))"
python -c "$QUOTE_CSV_PY" < file

To break it down:

• QUOTE_CSV_PY is a shell variable containing the Python one-liner commands
• The Python commands simply:
  • import the standard sys and csv modules;
  • create a csv writer that writes to standard output (stdout) with QUOTE_ALL set so all fields get quoted using quotechar, which is set to a single quote;
  • feed the csv writer a csv reader that reads from standard input (stdin).
• The second line simply passes the one-liner to the python interpreter, and feeds the csv file (called file) into its stdin.