Splitting different columns to different tables

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2008 Forums

SSIS and Import/Export (2008)

Splitting different columns to different tables

Author

Topic

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-17 : 11:00:10

Hi all, educational question only

I've got the connection manager prepped for a flat file, but need to output to THREE tables, which component do I need for this?

Jake

X002548
Not Just a Number

15586 Posts

Posted - 2012-02-17 : 11:36:15

I'm Guessing you are talking about SSIS

I would use bcp to a staging table, perform data audits and cleansing, the perform Del;at processing against the three tables using T-SQL...but that's just me ---T.REX

Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-17 : 11:52:53

I'm guessing you mean the Bulk Insert task.

Del; ??

X002548
Not Just a Number

15586 Posts

Posted - 2012-02-17 : 13:00:41

Yes or T-SQL BULK INSERT..I was talking about bcp (bulk copy program)

How big is the file and how many columns

Do you have a record layout?

Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-17 : 14:30:55

Name DOB Country Address 1 Address 2 Address 3 PostCode Total Spend $
Jane Doe 05-19-2000 United Kindgom 15 High Street London W2 3PX -1584.55

About 10 rows in total.

X002548
Not Just a Number

15586 Posts

Posted - 2012-02-17 : 14:39:59

CREATE A Staging Table to hold the data

THEN Do BULK INSERT INTO That table

Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-02-17 : 14:58:33

quote:
Originally posted by Jake Shelton

Hi all, educational question only

I've got the connection manager prepped for a flat file, but need to output to THREE tables, which component do I need for this?

Jake

you can use multicast to create a copies of resultset and link it to various outputs and map only required columns.

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-18 : 12:36:56

quote:
Originally posted by visakh16

quote:
Originally posted by Jake Shelton

Hi all, educational question only

I've got the connection manager prepped for a flat file, but need to output to THREE tables, which component do I need for this?

Jake

you can use multicast to create a copies of resultset and link it to various outputs and map only required columns.

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Thanks, I added a Flat File source and linked it to the Connection Manager, then linked the source to a new Multicast component, but it isn't picking up the input columns?

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-02-20 : 12:07:40

why? did you try connecting multicast output to destination and checking metadata?

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-22 : 13:24:44

Hi Visakh, yes that did the trick, but I have another question.

I need to ensure that the 3 destination tables have ID fields (probably best to use PK's), but the only option in the SS Destinations is a small 'Keep Identity' tickbox, which I don't think is enough.

Should I configure these in SSIS? Or do I need to set the tables up in Studio?

Edit - I just realised I can use an Execute SQL task to drop/recreate the tables. That's what I get for not clicking on the 'Control Flow' tab.

:)

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-22 : 19:14:56

Hi all, I just wanted to see if I'm going the right way.....

In the Control Flow tab:

Bulk Insert Task - this copies everything from a text file into a new staging table;
Execute SQL - On success of above, this performs data cleansing (I've yet to add code to it)

Data Flow tab:

Ole DB Source - Connects to the newly cleansed data;
Multicast - On success of above, splits out the appropriate columns to three destination tables (which already exist)
SQL Server Destination x 3 -

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-02-23 : 09:10:28

quote:
Originally posted by Jake Shelton

Hi all, I just wanted to see if I'm going the right way.....

In the Control Flow tab:

Bulk Insert Task - this copies everything from a text file into a new staging table;
Execute SQL - On success of above, this performs data cleansing (I've yet to add code to it)

Data Flow tab:

Ole DB Source - Connects to the newly cleansed data;
Multicast - On success of above, splits out the appropriate columns to three destination tables (which already exist)
SQL Server Destination x 3 -

sounds fine
only clarification needed is where are destination dbs. Are they in same machine where SSIS runs? if not,you've to use OLEDB destination instead

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Jake Shelton
Yak Posting Veteran

74 Posts

Posted - 2012-02-24 : 06:19:02

For educational purposes, yes the SS destinations are on the same machine, but I take your point. I should add that now I need to ensure the destinations drop/recreate the SQL tables and autopopulate an ID column upon every iteration, then export the data with the new ID's to Excel sheets.

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources