Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 SSIS and Import/Export (2008)
 Import/Export skipping last row

Author  Topic 

benking9987
Posting Yak Master

124 Posts

Posted - 2012-02-08 : 18:23:41
I am importing a .CSV file (actually multiple csv files within a folder at a time). For some reason, if the file has 4 rows in it (not including the header), then my SSIS package will only import 3. If the file has 250 rows in it, it will only import the first 249.

it doesn't appear to be a problem with header row recognition, its just skipping the last row in the file. If i open any of the .csv files and at a hard return at the very end of the file, then it will pick up the last row. otherwise it will not.

Is there something that I can do to my SSIS package to eliminate this problem? I don't have the capability to open every file and put a hard return at the end of each file.

Thanks all in advance.

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2012-02-08 : 18:29:43
Sounds like your files aren't being created correctly. Can you fix whatever process is creating the files?

Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page

benking9987
Posting Yak Master

124 Posts

Posted - 2012-02-08 : 18:32:11
They're generated for me and I have no control over the source. its a major ecommerce retailer thats giving them to me.
Go to Top of Page

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2012-02-08 : 18:37:32
Maybe write a powershell script to fix the files then. You could add a step to the package to run said script or have some external process periodically check and fix files as needed.

Have you tried a hex editor to verify what weird characters are in there to cause this? If you figured that out, you could tell the SSIS package to use a different row terminator/EOF character.



Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page

benking9987
Posting Yak Master

124 Posts

Posted - 2012-02-08 : 18:50:56
I did have it set up at a CRLF row delimiter and header delimiter. I opened the data in notepad++ and found that each row delimiter is actually only a LF.

I changed the setup within the SSIS package, but I'm still seeing the same results.

I could try a powershell script as you suggested, but I'm afraid that is outside of my experience range. Any good documentation you can point me to?
Go to Top of Page

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2012-02-08 : 18:57:26
Try these:

http://www.brangle.com/wordpress/2009/08/append-text-to-a-file-using-add-content-in-powershell/
http://www.computing.net/answers/programming/insert-a-carriage-return-into-a-txt-file/21856.html

Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page
   

- Advertisement -