SSIS reading LF as terminator when its set as CRLF

Question

using SSIS 2012. My flat file connection manager I have a delimited file where the row delimiter is set to CRLF, but when it processes the file, I have a text c

Answer 1

I had a similar issue to this. I had a CSV file with LF as the terminator. However, the client also had CRLF in two of the columns and this was causing the "delimiter for column is not found" error.

It took me a few days of googling solutions and trial and error, but I got it working.

In the end, I needed two script components.

In the first Script component, I had a column named Output0 string with Length of 4000. In the script (see below) I used ReadToEnd to load the data, replace the CRLF with an empty string, and then spliting into rows with the LF as the terminator.

using System.IO;
using System.Text;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    private StreamReader textReader;
    private string collateralFile;

public override void AcquireConnections(object Transaction)
{

    IDTSConnectionManager100 connMgr = this.Connections.Collateral;
    collateralFile = (string)connMgr.AcquireConnection(null);

}

public override void PreExecute()
{
    base.PreExecute();


}

public override void CreateNewOutputRows()
{

    StreamReader textReader = new StreamReader(collateralFile);
    string collatFile = textReader.ReadToEnd();


    collatFile = collatFile.Replace("\r\n", " ");
    
    String[] lines = collatFile.Split(new char[] { '\n' });
    textReader.Close();


        string nextLine;



        for (int i = 0; i < lines.Length; i++)
        {
            if (lines[i] != null)
            {
                nextLine = lines[i];

                if (!String.IsNullOrEmpty(nextLine))
                {
                    Output0Buffer.AddRow();
                    Output0Buffer.Output0 = nextLine;

                }
            }
          }

        }
    }

I tried splitting it again into columns, but it returned null values, so in the second script component I created my columns and loaded the data into them in the script.

public override void Input0_ProcessInputRow(Input0Buffer Row)
{

String[] columns = Row.Output0.Split(',');

Row.Description = columns[0];
Row.LegalDescription = columns[1];
Row.Address1ParsedLine1 = columns[2];
Row.Address1ParsedLine2 = columns[4];
Row.Address1ParsedCityname = columns[5];
Row.Address1ParsedStatecode = columns[6];
Row.Address1ParsedPostalcode = columns[7];
}