SQL Server BCP Bulk insert Pipe delimited with text qualifier format file

Question

I have a csv file which is vertical pipe delimited with every column also with a text qualifier of \".

I have been trying for ages to try and get the BCP format file

Answer 1

First, the BCP program only recognizes the doublequote as the container of the delimiter. So, using the single quote is causing an error.

Second, since the delimiter you want to specify "|" includes the doublequote character that BCP requires you to use to use to enclose your delimiter, you have to use the escape character to get the BCP program to ignore the quotes you want to use as delimiters. The escape character is the backslash character. So...

Instead of "|"... use... "\"|\""

This will tell BCP to ignore the doublequots preceded by a backslash and just treat them as any other character.

Third, you must account for the first field that has a preceding doublequote. The "|" terminator I mentioned above wont account for the opening doublequote at the beginning of each line for the first field.

To handle that you must add a "dummy" field to your format file and assign it's terminator as \" (or actually "\"" in the format file). Then, since you now have one more field in the file than you have in the table, you must offset your column numbering to tell BCP skip this new field that is terminated by the first doublequote in the file

Last, the last field is not terminated by just a newline character "\n". It is also termed by a doublequote (no pipe character included). So, we must customimze the final field terminator (which is actually the line/row terminator). Like this "\"\n".

Your format file will now look like this:

14.0
5
1   SQLCHAR   0  0   "\""  0  dummy_field   ""
2   SQLCHAR   0  0   "\"|\""  1  ID      ""
3   SQLCHAR   0  0   "\"|\""  2  SUB_ID  ""
4   SQLCHAR   0  0   "\"|\""  3  CODE1   SQL_Latin1_General_CP1_CI_AS
5   SQLCHAR   0  0   "\"|\""  4  CODE2   ""
6   SQLCHAR   0  0   "\"\n" 5  DATE    ""

I hope that helps.

Answer 2

What worked for me was changing

ROWTERMINATOR = '\n'

to

ROWTERMINATOR = '0x0a'.

Simply executing something like this if a rewrite it to your case:

BULK INSERT [dbo].[TEST]
FROM 'G:\DATA\TABLE.csv' 
WITH 
(   
    FIRSTROW = 2 
    , FIELDTERMINATOR ='|'
    , ROWTERMINATOR = '0x0a'
);