Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
rojak9999
Starting Member
3 Posts |
 Posted - 2009-12-03 : 22:56:54
|
| Hi all,I am new to this forum.I run into some memory problem when using new MERGE transformation in SSIS to refresh load a table in staging area from 2 large tables, each 1.1 mils record with 100+ columns each row.I have 2GB space left on hard drive, and when the package is run, it eat up all the space within 45 secs, then throw run out of memory error for the package.my package setting is:(1) source 1 (sorted using group by)--> 1.1 mil record (with updated data overnight)(2) source 2 (sorted using group by)--> 1.1 mil record(3) MERGE transformation --> merge to source 2(4) OLE DB destinationi also tried the following setting:(1) source 1 (not sorted) --> SORT transformation(2) source 2 (not sorted) --> SORT transformation(3) MERGE transformation --> merge to source 2(4) OLD DB destinationthis setting chew up 2GB of space in secs.Is there any alternative method to decrease memory consumption when using MERGE transformation in SSIS package for ETL?Thanks. |
|
|
|
|
|
MERGE transformation - Memory Problem