CSV.read Illegal quoting in line x

Question

I am using ruby CSV.read with massive data. From time to time the library encounters poorly formatted lines, for instance:

\"Illegal quoting in line 53657.\"         


        

Answer 1

The liberal_parsing option is available starting in Ruby 2.4 for cases like this. From the documentation:

When set to a true value, CSV will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields.

To enable it, pass it as an option to the CSV read/parse/new methods:

CSV.read(filename, liberal_parsing: true)