Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
parody
Posting Yak Master
111 Posts |
 Posted - 2011-02-25 : 11:30:28
|
| HiWe have some relatively large files to import, some much larger than others. The worst will contain circa 1million rows a day and is 1000 odd columns wide.These are diff files so they contain an entire line if any one column changes. The business want to be able to track the data position for any dayfor example:Position FileDate20110101 20110101 New File20110102 20110101 Old File20110103 20110101 Old File20110104 20110104 New File20110105 20110105 New File20110106 20110105 Old FileSo the positon reflects the most recent file received for a given primary key. We also need to track what the current at month end. It could be that a row does change for months, so would cross month ends.Currently I am doing this 1 of 3 ways I thought of, which during testing seemed to be the best performance.Without clouding your minds with those ways, can anyone suggest some methods of doing this? |
|
|
parody
Posting Yak Master
111 Posts |
Posted - 2011-02-25 : 11:43:59
|
| make that 4 ways, just going to test another. |
 |
|
|
parody
Posting Yak Master
111 Posts |
Posted - 2011-02-25 : 11:47:50
|
| Actually I will tell you them else i'll get replys for what I have already done!1) re-insert each row every day, changed or otherwise2) seperate table to track guid of what is current on each day, join back to materialise additional rows3) static date table, view which gets max row <= static date4) cross apply a function to perform simlar to 3 |
|
|
|
|
|
|
Large Import & tracking suggestions