There are two ways to achieve this.
1) We can move the flat files and then rename it.
2) While moving files itself, automatic rename should be done.
We will do the second type. The criteria is to rename the files while moving from source to destination.
So for that, we need FILE SYSTEM TASK to be included. Secondly since we need to move many files, we will use FOR EACH LOOP CONTAINER.
To fetch all the files, we can use FOR EACH LOOP task in SSIS.
In collection tab, we can select FOREACH FILE enumerator option for fetching files and we can change enumerator configuration
Folder option: Points to source where we need to fetch files.
Files: will give us idea whether we need to fetch all the files (*.*) or if we give extension like *.txt, it is going to fetch only .txt files .
Once I give Source name in FOR EACH LOOP container, It is going to fetch all the files corresponding to that path.
Retrieve file name:
This option is used to let the variables mentioned in VARIABLE MAPPING tab, like how fetch the file and in what format.
This is supported by 3 options:
1) Fully qualified
2) Name and extension
3) Name only
1) Fully qualified: Fully qualified will assign path and file name with its extension to variable present in VARIABLE MAPPING.
2) Name and extension: If we use name and extension then, Only name and extension will be assigned to variable at VARIABLE MAPPING option
3) Name Only: Name only option enables variable at VARIABLE MAPPING option to fetch only name of file (Not even the extension)
Say example: If we have three text files in our source like txt1, txt2, txt3 and we need to fetch all three text files.
Now these files which are actually my source are in the path C:\Documents and Settings\swami\Desktop\test\source
Now I want to move these files to C:\Documents and Settings\swami\Desktop\test\dest.
My requirement is that while moving these files, I should rename these files and should place to destination folder 'Dest'.
Now at FOR EACH LOOP container, We will give source path where our files resides at Folder option from collection tab.
secondly I'm going to give *.txt in files position to retrieve only .txt files.
Retrieve file name is selected as Name and extension and only name and extension will be passed to any variable I assign to VARIABLE MAPPING tab.
Now I'm assigning a dummy variable named by me as 'Filename' to VARIABLE MAPPING tab.
Now variable Filename will take only Name of file and extension from given path.
ie,. Values of Filename taken by foreach loop container at first iteration during run will be only txt1.txt
and on second iteration, it will be txt2.txt and so on.. Iteration counts are handled by foreach loop container internally.
now for renaming, Im going to create one more variable with values being the path of source. ie., I create a variable named 'source' and value is C:\Documents and Settings\swami\Desktop\test\source.\
|||rly Im creating one more variable for destination named 'destination' and values assigned will be C:\Documents and Settings\swami\Desktop\test\dest.
Now I'm adding a FILE SYSTEM TASK inside foreach loop container.
I'm creating one more variable called 'sourcepath'.
Reason for creating this variable is that 'souce' variable assign only source path name and filename variable have only files name with its extension.
to integrate path and then corresponding file loop fetches with extension, I created this variable.
The expression for variable sourcepath is given as user::source+user::filepath.
ie,. on first iteration, Sourcepath will have value as C:\Documents and Settings\swami\Desktop\test\source\txt1.txt.
and on second iteration, it will have values like C:\Documents and Settings\swami\Desktop\test\source\txt2.txt. and so on..
I'm assigning this variable Source path to File System task in source column.
Similar things are done on destination also by creating new variable 'destinationpath' where its expression is given as user::destination+'newname_'+user::filepath
where filepath is txt1.txt, txt2.txt and so on variable destination is indicates to destination path.
After running package, The files are transferred with their name are changed. The complete snippets are shown follows:-
1) We can move the flat files and then rename it.
2) While moving files itself, automatic rename should be done.
We will do the second type. The criteria is to rename the files while moving from source to destination.
So for that, we need FILE SYSTEM TASK to be included. Secondly since we need to move many files, we will use FOR EACH LOOP CONTAINER.
To fetch all the files, we can use FOR EACH LOOP task in SSIS.
In collection tab, we can select FOREACH FILE enumerator option for fetching files and we can change enumerator configuration
Folder option: Points to source where we need to fetch files.
Files: will give us idea whether we need to fetch all the files (*.*) or if we give extension like *.txt, it is going to fetch only .txt files .
Once I give Source name in FOR EACH LOOP container, It is going to fetch all the files corresponding to that path.
Retrieve file name:
This option is used to let the variables mentioned in VARIABLE MAPPING tab, like how fetch the file and in what format.
This is supported by 3 options:
1) Fully qualified
2) Name and extension
3) Name only
1) Fully qualified: Fully qualified will assign path and file name with its extension to variable present in VARIABLE MAPPING.
2) Name and extension: If we use name and extension then, Only name and extension will be assigned to variable at VARIABLE MAPPING option
3) Name Only: Name only option enables variable at VARIABLE MAPPING option to fetch only name of file (Not even the extension)
Say example: If we have three text files in our source like txt1, txt2, txt3 and we need to fetch all three text files.
Now these files which are actually my source are in the path C:\Documents and Settings\swami\Desktop\test\source
Now I want to move these files to C:\Documents and Settings\swami\Desktop\test\dest.
My requirement is that while moving these files, I should rename these files and should place to destination folder 'Dest'.
Now at FOR EACH LOOP container, We will give source path where our files resides at Folder option from collection tab.
secondly I'm going to give *.txt in files position to retrieve only .txt files.
Retrieve file name is selected as Name and extension and only name and extension will be passed to any variable I assign to VARIABLE MAPPING tab.
Now I'm assigning a dummy variable named by me as 'Filename' to VARIABLE MAPPING tab.
Now variable Filename will take only Name of file and extension from given path.
ie,. Values of Filename taken by foreach loop container at first iteration during run will be only txt1.txt
and on second iteration, it will be txt2.txt and so on.. Iteration counts are handled by foreach loop container internally.
now for renaming, Im going to create one more variable with values being the path of source. ie., I create a variable named 'source' and value is C:\Documents and Settings\swami\Desktop\test\source.\
|||rly Im creating one more variable for destination named 'destination' and values assigned will be C:\Documents and Settings\swami\Desktop\test\dest.
Now I'm adding a FILE SYSTEM TASK inside foreach loop container.
I'm creating one more variable called 'sourcepath'.
Reason for creating this variable is that 'souce' variable assign only source path name and filename variable have only files name with its extension.
to integrate path and then corresponding file loop fetches with extension, I created this variable.
The expression for variable sourcepath is given as user::source+user::filepath.
ie,. on first iteration, Sourcepath will have value as C:\Documents and Settings\swami\Desktop\test\source\txt1.txt.
and on second iteration, it will have values like C:\Documents and Settings\swami\Desktop\test\source\txt2.txt. and so on..
I'm assigning this variable Source path to File System task in source column.
Similar things are done on destination also by creating new variable 'destinationpath' where its expression is given as user::destination+'newname_'+user::filepath
where filepath is txt1.txt, txt2.txt and so on variable destination is indicates to destination path.
After running package, The files are transferred with their name are changed. The complete snippets are shown follows:-
Hi Dhinakaran S,
ReplyDeleteGreat article,
and how to do the file name extend with date. In above example txt1_2012-05-10.txt,txt2_2012-05-10.txt and txt3_2012-05-10.txt.
This is an urgent issue, Please anyone could help me on this.
Thanks
Anil
How To Move Multiple Files In Ssis And Also Rename Simultaneously >>>>> Download Now
Delete>>>>> Download Full
How To Move Multiple Files In Ssis And Also Rename Simultaneously >>>>> Download LINK
>>>>> Download Now
How To Move Multiple Files In Ssis And Also Rename Simultaneously >>>>> Download Full
>>>>> Download LINK W1
Hello Anil.
ReplyDeleteIn Foreach loop editor, Under Retrieve Filename, instead of Name and Extension, you can use Name Only and user::filename variable will hold only the name of file.
Now use this variable to append the date and extension at last.
Example: if ur filename is abc.txt, then variable user::filename will hold 'abc' and then in FILESYSTEM task,
Use this to append the date function as I explained earlier in same post with Extension ('.txt' in your case).
Hope you are cleared.
I think I'm about to go crazy! Thanks so much for the example... I've replicated it (3 times from scratch so far)...but it won't run successfully. Here is the error I get. Any idea what I'm doing wrong? Oh... so frustrated.
ReplyDeleteTITLE: Package Validation Error
------------------------------
Package Validation Error
------------------------------
ADDITIONAL INFORMATION:
Error at File System Task: Failed to lock variable "D:\SSIS Jobs\Jobs_Prepense\Example SSIS Jobs\Move_RenameFiles\Source\" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".
Error at File System Task [File System Task]: An error occurred with the following error message: "Failed to lock variable "D:\SSIS Jobs\Jobs_Prepense\Example SSIS Jobs\Move_RenameFiles\Source\" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".
".
Error at File System Task: Failed to lock variable "D:\SSIS Jobs\Jobs_Prepense\Example SSIS Jobs\Move_RenameFiles\dest\_" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".
Error at File System Task [File System Task]: An error occurred with the following error message: "Failed to lock variable "D:\SSIS Jobs\Jobs_Prepense\Example SSIS Jobs\Move_RenameFiles\dest\_" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".
".
Error at File System Task: There were errors during task validation.
(Microsoft.DataTransformationServices.VsIntegration)
------------------------------
BUTTONS:
OK
------------------------------
Hello Jim,
DeleteThis error means, that you have not configured your variable properly in FILE SYSTEM task. Please check the issue what variable you have used.
I am also facing same error and couldn't resolve it.
DeleteNice information. Thanks for sharing content and such nice information for me. I hope you will share some more content about. Please keep sharing!
ReplyDeletebig data training in chennai
Your blog is very interesting. And the point describing the information very effectively.
ReplyDeleteBig Data Institutes in Pune
Thanks for sharing the interesting post. Hope you share more post like this.
ReplyDeleteBigdata training in Pallikranai
Machine learning training in Pallikranai
Spark training in Pallikranai
Data analytics training in Pallikranai
Data science training in Pallikranai
Spark with ML training in Pallikranai
Python training in Pallikranai
MongoDB training in Pallikaranai
Hadoop training in Pallikranai
Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
ReplyDeleteBest PHP Training Institute in Chennai|PHP Course in chennai
Best .Net Training Institute in Chennai
Software Testing Training in Chennai
Blue Prism Training in Chennai
Angularjs Training in Chennai
I think there are actually more and more useful aspects of SSIS and has more functions in order to solve complex IT problems.
ReplyDeleteSSIS Upsert
Thanks for the nice Post. I must suggest your readers to Visit Big data course in coimbatore
ReplyDeletewith us to get some idea about that content.Thanks for sharing
ReplyDeleteMsbi Online Training in Hyderabad
Msbi Online Training in India
How To Move Multiple Files In Ssis And Also Rename Simultaneously >>>>> Download Now
ReplyDelete>>>>> Download Full
How To Move Multiple Files In Ssis And Also Rename Simultaneously >>>>> Download LINK
>>>>> Download Now
How To Move Multiple Files In Ssis And Also Rename Simultaneously >>>>> Download Full
>>>>> Download LINK X2