process the new files only

Question

I have a source from which the files are to be processed. Multiple files are coming to that loacation at any time randomly (Package should run every 2 hours). I have to process

Answer 1

You can achieve this using the following steps.

1. Use the foreach file enumerator for your incoming folder and save the filename in "IncomingFile" variable. Configure to select "Name and Extension"[In my code I have used that otherwise you need to do some modification to the script]
2. Create tow SSIS variables Like "ArchivePath" as string and "IsLoaded" as Boolean[default to false].
3. Create the SSIS script component and use "IncomingFile" and "ArchivePath" as the readonly variable. "IsLoaded" should be the ReadandWrite variable.

4. Write the following code in the script component. If file is already exists then it will return true. Otherwise False.

   public void Main()
   {
       var archivePath = Dts.Variables["ArchivePath"].Value.ToString();
       var incomingFile = Dts.Variables["IncomingFile"].Value.ToString();
   
       var fileFullPath = string.Format(@"{0}\{1}",archivePath,incomingFile);
   
       bool isLoaded = File.Exists(fileFullPath);
   
       Dts.Variables["IsLoaded"].Value = isLoaded;
   
       Dts.TaskResult = (int)ScriptResults.Success;
   }
   

5. Use the Precedence constraint to call the Data flow task and evaluation operation should be "Expression" . Set something as follows in your expression box.

   @IsLoaded==False

Hope this helps.