Fetch Incremental data on SAP tables using SSIS

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2008 Forums

Transact-SQL (2008)

Fetch Incremental data on SAP tables using SSIS

Author

Topic

kond.mohan
Posting Yak Master

213 Posts

Posted - 2013-06-03 : 03:59:12

Hi all,
we have created SSIS packages for Data loading from Source database(SAP Tables) to Sql server 2012(Stagingdatabase) on DAILY Basis(using SQL Agent job).few of tables having huge amount of data,those tables were consuming much time for Data loading.(SAMPLE table name :KONV,VBFA,MSEG....)we need to incremental load of those tables (instead of full truncate and full load).

Is there any way to load only incremental for few critical tables?

Suggestions are Appreciated.

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2013-06-03 : 04:17:00

yep. do you've a audit field like datecreated or modified date in your tables?

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/
https://www.facebook.com/VmBlogs

kond.mohan
Posting Yak Master

213 Posts

Posted - 2013-06-05 : 06:11:25

Hi vishak,
thanks for reply.we have Primary key(combination of columns) in specified table but we don't have updation and modified datetimefiled in the KONV Table.for this updation and modification fields purpose
i have another table called vbrk (vbrk.knumv=konv.knumv join condition)
vbrk table have erdat,erzet(datemodification) are modification fields.

so we have fetched incremental based on those fiedls
table 1 ---konv
table 1 pk---kv.knumv+cast(kv.kposn as varchar)+cast(kv.STUNR as varchar)+cast(zaehk as varchar)
table 2 ---vbrk
table 2 pk--mandt+vbeln

so pls guide on this table incremental process

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2013-06-05 : 06:15:28

then what you can do is have a control table to store each days max(erzet) value from vbrk.
The etl process will fetch the latest record from control table and get its stored max(erzet) value in a parameter (sqy @fetcheddatevalue).
this will then be used in sql statement as


SELECT requiredcolumns...
FROM KONV k
JOIN vbrk v
On v.knumv=k.knumv
WHERE v.erzet > @fetcheddatevalue

this will give you the increemetal data which you can use in MERGE or do update/insert to get it reflected in target table

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/
https://www.facebook.com/VmBlogs

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources