Importing Grouped Report Data to Database

Question

My company receives data from a client that is unable to provide data in any direct format so we have to import several reports that are in a grouped layout like the one bel

Answer 1

I'd do it with a script component.

Total Data Flow:

ExcelSource --> Script Component (Tranformation) --> Conditional Split --> SQL Destination

In script component:

Check accountSummary on InputColumns

Add ActivityDate as output column.

Open Script:

outside of your row processing.

Add:

public datetime dte;

inside row processing:

if (DateTime.TryParse(Row.ActivitySummary.ToString()))
{dte=DateTime.Parse(Row.ActivitySummary.ToString());}
else
{Row.ActivityDate = dte;}

Then add a Conditional Split to remove null Activity Dates

Answer 2

Hmm - well you will definitely have to do some programmatic adjusting of the data set to add that group date to the detail line. I'm unsure of how you will be importing the xlsx but I would recommend first off just using a SSIS package and then doing the adjustments in a script task as the "best" way to do this. See here on how to handle Excel in SSIS Script tasks.

If you don't know SSIS or especially programming though, you're next best bet (in my opinion) is to just import the data into a staging table, do the manipulations with T-SQL and then insert that table into your main table. I did a SQL Fiddle of this here.

CREATE TABLE ActivitySummary
(
  id int identity(1,1),
  activity_date date,
  activity varchar(100),
  paid_time decimal(5,2),
  unpaid_time decimal(5,2),
  total_time decimal(5,2)
)

CREATE TABLE ActivitySummary_STG
(
  id int identity(1,1),
  activity_date date,
  activity varchar(100),
  paid_time decimal(5,2),
  unpaid_time decimal(5,2),
  total_time decimal(5,2)
)
GO

-- Simulate import of Excel sheet into staging table
truncate table ActivitySummary_STG;
GO

INSERT INTO ActivitySummary_STG (activity_date, activity, paid_time, unpaid_time, total_time)
select '8/14/17',null,null,null,null 
UNION ALL
select null,'001 Lunch',0,4.4,4.4
UNION ALL
select null,'002 Break',4.2,0,4.2
UNION ALL
select null,'007 System Down',7.45,0,7.45
UNION ALL
select null,'019 End of Work Day',0.02,0,0.02
UNION ALL
select '8/15/17',null,null,null,null 
UNION ALL
select null,'001 Lunch',0,4.45,4.45
UNION ALL
select null,'002 Break',6.53,0,6.53
UNION ALL
select null,'007 System Down',0.51,0,0.51
UNION ALL
select null,'019 End of Work Day',0.02,0,0.02
GO

-- Code to massage data
declare @table_count int = (select COALESCE(count(id),0) from ActivitySummary_STG);
declare @counter int = 1;

declare @activity_date date,
        @current_date date;

WHILE (@table_count > 0 AND @counter <= @table_count)
BEGIN 
  select @activity_date = activity_date
  from ActivitySummary_STG
  where id = @counter;

  if (@activity_date is not null)
    BEGIN
      set @current_date = @activity_date;

      delete from ActivitySummary_STG
      where id = @counter;
    END
  else
    BEGIN
      update ActivitySummary_STG SET
        activity_date = @current_date
      where id = @counter;
    END

  set @counter += 1;
END

INSERT INTO ActivitySummary (activity_date, activity, paid_time, unpaid_time, total_time)
select activity_date, activity, paid_time, unpaid_time, total_time 
from ActivitySummary_STG;

truncate table ActivitySummary_STG;
GO

select * from ActivitySummary;