Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 SSIS and Import/Export (2005)
 Problem with too many lookups!!

Author  Topic 

Ali Chadorbaf
Starting Member

44 Posts

Posted - 2008-02-28 : 12:40:55
Hi,
My process passing 1,000,000 rows to a data flow with about 20 lookups to get the keys that I wantted .
Most lookups have small number of rows except one with over 5,000,000 rows. I cannot get the process to run (the process hanged) probably because of memory issue. Any clue where/how I can tune it.
Thanks

mwjdavidson
Aged Yak Warrior

735 Posts

Posted - 2008-03-03 : 05:36:10
Is the lookup fully cached? Are you using a query (rather than a table) for the lookup? If so, make sure this is as narrow as possible (i.e. just natural key and surrogate key) and think about how much memory you have available. Do you need to return all those rows in the lookup? Is there, for example, historical information in the table that you can exclude for lookup purposes.
You may well have to switch to partial caching with a lookup table that size.

Mark
Go to Top of Page
   

- Advertisement -