Regular expression to match CSV delimiters

Question

I\'m trying to create a PCRE that will match only the commas used as delimiters in a line from a CSV file. Assuming the format of a line is this:

1,\"abcd\",         


        

Answer 1

As you've already been told, a regular expression is really not appropriate; it is tricky to deal with the general case (doubly so if newlines are allowed in fields, and triply so if you might have to deal with malformed CSV data.

• I suggest the tool CSVFIX as likely to do what you need.

To see how bad CSV can be, consider this data (with 5 clean fields, two of them empty):

"""",,"",a,"a,b"

Note that the first field contains just one double quote. Getting the two double quotes squished to one is really rather tough; you probably have to do it with a second pass after you've captured both with the regex. And consider this ill-formed data too:

"",,"",a",b c",

The problem there is that the field that starts with a contains a double quote; how to interpret it? Stop at the comma? Then the field that starts with b is similarly ill-formed. Stop at the next quote? So the field is a",b c" (or should the quotes be removed)? Etc...yuck!

This Perl gets pretty close to handling correctly both the above lines of data with a ghastly regex:

use strict;
use warnings;

my @list = ( q{"""",,"",a,"a,b"}, q{"",,"",a",b c",} );

foreach my $string (@list)
{
    print "Pattern: <<$string>>\n";
    while ($string =~ m/ (?: " ( (?:""|[^"])* ) "  |  ( [^,"] [^,]* )  |  ( .? ) )
                         (?: $ | , ) /gx)
    {
        print "Found QF: <<$1>>\n" if defined $1;
        print "Found PF: <<$2>>\n" if defined $2;
        print "Found EF: <<$3>>\n" if defined $3;
    }
}

Note that as written, you have to identify which of the three captures was actually used. With two stage processing, you could just deal with one capture and then strip out enclosing double quotes and nested doubled up double quotes. This regex assumes that if the field does not start with a double quote, then there double quote has no special meaning within the field. Have fun ringing the changes!

Output:

Pattern:  <<"""",,"",a,"a,b">>
Found QF: <<"">>
Found EF: <<>>
Found QF: <<>>
Found PF: <>
Found QF: <>
Found EF: <<>>
Pattern:  <<"",,"",a",b c",>>
Found QF: <<>>
Found EF: <<>>
Found QF: <<>>
Found PF: <>
Found PF: <>
Found EF: <<>>

We can debate whether the empty field (EF) at the end of the first pattern is correct; it probably isn't, which is why I said 'pretty close'. OTOH, the EF at the end of the second pattern is correct. Also, the extraction of two double quotes from the field """" is not the final result you want; you'd have to post-process the field to eliminate one of each adjacent pair of double quotes.