Import the .log files by skipping the last file

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2008 Forums

SSIS and Import/Export (2008)

Import the .log files by skipping the last file

Author

Topic

kareena
Starting Member

1 Post

Posted - 2009-11-03 : 14:09:22

I am new to SSIS and it would be great if someone could help me.

I am using an SSIS Package to import .log files into a database. The format of the log files are yyyymmddXYZ.log.

My SSIS Package consists of the following:
1. Execute SQL Task that truncates the destination (database) rows.
2. Data Flow Task inside a ForEach Loop
3. Data Flow Task Consists of Flate File Source,Derived Column and OLE Db Destination.

i am getting this error at Flat File Source when it tries to pull data from "today's" file "[Flat File Source [1]] Warning: The process cannot access the file because it is being used by another process." for the last (11/03) .log file.
I think this is a valid message as there is another process that is updating "Today's" file. All the other (older than 11/02) .log files are getting imported into database without any errors.

http://www.sqlservercentral.com/blogs/tim_mitchell/archive/2009/9/16/skipping-items-in-a-foreach-loop.aspx

I used the above link to skip the latest (today's) .log file.

I tried creating a variable(varFile):

@[User::FilePath] +
+ RIGHT((DT_WSTR,4)YEAR(GETDATE()),2) + (DT_WSTR,2)MONTH(GETDATE()) + (DT_WSTR,2)DAY(GETDATE()) + "XYZ" + ".log")

\\servername\logfiles\20091103XYZ.log

I put this variable in the Precedence COnstraint Editor as @[User::varFile] != "yes".

This doesnt seem to be working.i could be totally wrong. Can someone help me with the expression.

I need an expression to skip the "today's" .log file and i will place it in a Precedence Constraint (as per the blog)

Any help would be great.

YellowBug
Aged Yak Warrior

616 Posts

Posted - 2009-11-05 : 06:15:07

In the Precedence COnstraint Editor change to:
[Variable name for the files in the loop] !=
@[User::FilePath] +
+ RIGHT((DT_WSTR,4)YEAR(GETDATE()),2) + (DT_WSTR,2)MONTH(GETDATE()) + (DT_WSTR,2)DAY(GETDATE()) + "XYZ" + ".log")

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources