SSIS : Using multicast to enter data into 2 RELATED destinations

Question

I am new to SSIS. I have data coming from a single source. I need to enter that data into several tables (these tables are related by foreign key relationships). I am using multicast to enter the data into several destinations.

My question is...

How do I get the identity of an entry into one destination and use that identity for the foreign key column of the 2nd destination?

Here is an example of what I am looking for. The Employee table has a foreign key to the Address table. But the source includes all this information. Entering the data into 2 different locations is easy. But I need the identity from the Address table when I enter the info into the Employee table. How do I get that Id?

Source (Excel file)

Name        Street           State        etc...
----        ------           -----
Jim         12345 Plain St.  CA
Bob         54321 Main St.   CA
etc.

Destination

   Address        Employee
   -------        -------
PK AddressId   PK EmployeeId
   Street      FK AddressId
   State          Name
   etc...         etc...

Answer 1

The short answer is that, out of the box, SSIS isn't built for that. Once you land data in a table, the destination components don't allow for an output stream to be sent.

You could fake this behaviour out by using an OLE DB Command but your performance will be less than good since it will issue a singleton insert statement for every row that flows through the data flow. Generally, the engine attempts to batch N units of work up and perform bulk, set-based operations on the data to get more throughput. You could also use a Script Component to perform this. The same performance caveat would still apply.

A better option might be to land your data into a staging table and then use an Execute SQL Task after your Data flow and use the OUTPUT clause of the INSERT operation to capture those identities and then patch them into your other table.

Answer 2

Another approach I would try in with task like yours is to artificially generate the ID field for the parent table. The idea here is knowing the ID ahead so you can assign the foreign key values.

Then instead of using multicast, load the data sequentially: parent, and then child. For the parent table, tick the Keep Identity property (OLEDB Destination).

Answer 3

Staging table

CREATE TABLE [dbo].[Stage](
    [Name] [varchar](50) NULL,
    [Street] [varchar](50) NULL,
    [State] [varchar](50) NULL,
    [pkAddressID] [int] NULL
) ON [PRIMARY]

GO

Address table

CREATE TABLE [dbo].[Address](
    [pkAddressID] [int] IDENTITY(1,1) NOT NULL,
    [street] [varchar](50) NULL,
    [state] [varchar](50) NULL
) ON [PRIMARY]

-- Employee table

CREATE TABLE [dbo].[Employee](
    [pkEmployeeID] [int] IDENTITY(1,1) NOT NULL,
    [fkAddressID] [int] NULL,
    [Name] [varchar](50) NULL
) ON [PRIMARY]

GO

--- DFT - Load data into Stage table

--- Execute SQL Task 1 --- Populate Address table

Merge [dbo].[Address] as target
using
(
    select distinct [Street], [State] from [dbo].[Stage]
) as source
on  source.[Street] = target.[Street] and source.[State] = target.[State]

when not matched then
insert ([Street], [State])
values (source.[Street], source.[State])
;

--- Execute SQL Task 2 --- Populate Stage table//pkAddressID column

Merge [dbo].[Stage] as target
using
(
    select [pkAddressID],[Street], [State] from [1Address]
) 
as source 
on source.[Street] = target.[Street] and source.[State] = target.[State]

when matched then
update
set target.[pkAddressID] = source.[pkAddressID]
;

--- Execute SQL Task 3 --- Populate Employee table

Merge [dbo].[Employee] as target
using
(
    select [pkAddressID], [Name] from [dbo].[1Stage]
) as source
on source.[pkAddressID] = target.[fkAddressID]
and source.[Name] = target.[Name]

when not matched then
insert ([fkAddressID], [Name])
values (source.[pkAddressID], source.[Name])
;

Answer 4

The other way to overcome The INSERT statement conflicted with the FOREIGN KEY constraint ... error while inserting into two related tables with Multicast is to clear option Check constraints for dependent OLE DB Destination: