Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
Author |
Topic |
LearningSQLKid
Yak Posting Veteran
51 Posts |
Posted - 2014-04-06 : 09:19:35
|
Hi ExpertsI need to know what are the disadvantages of metadata driven etl framework.I have a metadata repository of all my objects and someone suggested me that i should go with the metadata driven approach and create ssis packages based on my metadata. This means that i have to find some programming tool like bidshelper also is it possible to create a packages which has columns stored in database with their datatypes and destination table column mappings .. does ssis allow this type of approach. pleased help me ..ThanksSelect Knowledge from LearningProcess |
|
gbritton
Master Smack Fu Yak Hacker
2780 Posts |
Posted - 2014-04-07 : 15:20:45
|
SSIS already maintains metadata and is metadata-driven out of the box. That's just how it works. I think you're asking if you can have another metadata source that SSIS can refer to. The short answer is not really. The long answer is that it should be possible to take in your metadata and produce SSIS-compliant XML that you can use to build SSIS packages on the fly (they're just XML). Personally I wouldn't want to do that, but it's up to you. |
|
|
LearningSQLKid
Yak Posting Veteran
51 Posts |
Posted - 2014-04-08 : 08:59:52
|
Thanks gbrittonHere is my problem and may be you can suggest some good pointsI have a database called MetaDataDB. This database has a table called ETLLoadTables. The structure is belowTableId , TableName, Query1 Customers Select CustomerID,CustomerName From Customers2 Sale Select SaleID,CustomerID From SaleNow I want to loop through this table get the Query In variable , pass this variable to source in dataflow and refresh the mappings and load it to destination .. is it possible to achieve in ssis 2008,2012 ?I can use it EXECUTE SQL task but how can i make data flow to work in dynamic fashion. any ideaThank you so much Select Knowledge from LearningProcess |
|
|
gbritton
Master Smack Fu Yak Hacker
2780 Posts |
Posted - 2014-04-08 : 14:20:26
|
quote: Originally posted by LearningSQLKid Thanks gbrittonHere is my problem and may be you can suggest some good pointsI have a database called MetaDataDB. This database has a table called ETLLoadTables. The structure is belowTableId , TableName, Query1 Customers Select CustomerID,CustomerName From Customers2 Sale Select SaleID,CustomerID From SaleNow I want to loop through this table get the Query In variable , pass this variable to source in dataflow and refresh the mappings and load it to destination .. is it possible to achieve in ssis 2008,2012 ?I can use it EXECUTE SQL task but how can i make data flow to work in dynamic fashion. any ideaThank you so much Select Knowledge from LearningProcess
Things will work up until the "refresh the mappings" part. I know of no way to dynamically tell SSIS to do that. Think about it for a minute. When you build a package in the SSIS designer, it in turn builds an XML file (the .dtsx -- x is for XML) that eventually gets run. At execution time, the SSIS engine validates the dtsx file. I think that you are basically asking the engine to modify the dtsx on the fly. I don't believe that it can do that. |
|
|
LearningSQLKid
Yak Posting Veteran
51 Posts |
Posted - 2014-04-10 : 07:47:23
|
quote: Originally posted by gbritton
quote: Originally posted by LearningSQLKid Thanks gbrittonHere is my problem and may be you can suggest some good pointsI have a database called MetaDataDB. This database has a table called ETLLoadTables. The structure is belowTableId , TableName, Query1 Customers Select CustomerID,CustomerName From Customers2 Sale Select SaleID,CustomerID From SaleNow I want to loop through this table get the Query In variable , pass this variable to source in dataflow and refresh the mappings and load it to destination .. is it possible to achieve in ssis 2008,2012 ?I can use it EXECUTE SQL task but how can i make data flow to work in dynamic fashion. any ideaThank you so much Select Knowledge from LearningProcess
Things will work up until the "refresh the mappings" part. I know of no way to dynamically tell SSIS to do that. Think about it for a minute. When you build a package in the SSIS designer, it in turn builds an XML file (the .dtsx -- x is for XML) that eventually gets run. At execution time, the SSIS engine validates the dtsx file. I think that you are basically asking the engine to modify the dtsx on the fly. I don't believe that it can do that.
Thanks gbrittonI too believe that it is not possible ... thank you so much for the reply and your timeSelect Knowledge from LearningProcess |
|
|
LFBI
Starting Member
1 Post |
Posted - 2014-05-05 : 21:24:20
|
I'm late to the conversation, but thought you may be interested in checking out LeapFrogBI.com. LeapFrogBI is a metadata driven ETL application which generates SSIS packages including dynamically updating field mapping. Drop me a line if you want to learn more. paul@leapfrogbi.com |
|
|
jeffw8713
Aged Yak Warrior
819 Posts |
|
|
|
|
|
|