none
Batching of files using FTP adapter RRS feed

  • Question

  • hi all,

    We have a requirement that BizTalk should pick only 5 files out of 50 files,at one batch using FTP adapter receive side. The other pick up of batch files should be after 5minutes. We tried to do this by updating the maximum files to 5 and the interval we set it to 5 minutes in the configuration of receive location.

    But to our surprise, its picking all the 50 files and sending instead of picking the files in a batch.

    Can anyone please help me in resolving this issue?

    Thanks in advance,

    Veena

    Tuesday, August 18, 2015 12:27 PM

Answers

  • Hi Veena,

    With receive location configuration its not possible to achieve your scenario, because RL will pick all the file at a time.

    I have faced same type of scenario in one project and I have achieved it with windows power-shell script and Task Scheduled. Let me share you power-shell script for your reference.

    #Get 'n' number of files, you can set number as per your requirement
    $FileLimit = 5 
    #Pick-up location for files
    $PickupDirectory = Get-ChildItem -Path "\\Location\FilePick\"
    #Destination location for files
    $DropDirectory = "\\Location\FileDrop\"
    $Counter = 0
    foreach ($file in $PickupDirectory)
    {
        if ($Counter -ne $FileLimit)
        {
            $Destination = $DropDirectory+$file.Name
            Write-Host $file.FullName #Output file fullname to screen
            Write-Host $Destination   #Output Full Destination path to screen
            Move-Item $file.FullName -destination $Destination
            $Counter++
        }  
    }

    Please configure above script to Task scheduler task and set duration 5 min. This process will help you to pick 5 files after every 5 min.


    Thanks, If my reply is helpful please mark as answer or vote as helpful.

    • Proposed as answer by [Kamlesh Kumar] Sunday, August 23, 2015 6:27 PM
    • Marked as answer by Angie Xu Wednesday, August 26, 2015 6:55 AM
    Wednesday, August 19, 2015 5:35 PM

All replies

  • What is the actual requirement here.  5 files every 5 minutes seems a bit arbitrary.

    Either way, this is not possible with the FTP Adapter.

    A Windows Scheduler Task with a client FTP Script is a good alternative.

    Tuesday, August 18, 2015 5:00 PM
  • Hi Veena,

    You can use Directory.GetFiles method to get the batch of files through .Net component and then drop the batched files in the desired location were BizTalk FTP adapter can poll the files . This is the simplest approach to go ahead with your task . FTP adapter  does not have capability to batch and pick  files on arbitrary basis .

    You can refer http://stackoverflow.com/questions/7039580/multiple-file-extensions-searchpattern-for-system-io-directory-getfiles for your case.

    Thanks

    Abhishek

    • Marked as answer by Veena H Wednesday, August 19, 2015 5:28 AM
    • Unmarked as answer by Veena H Wednesday, August 19, 2015 5:49 AM
    Wednesday, August 19, 2015 5:09 AM
  • Thank you Abhishek. We will try this.
    Wednesday, August 19, 2015 5:28 AM
  • Hi Veena,

    With receive location configuration its not possible to achieve your scenario, because RL will pick all the file at a time.

    I have faced same type of scenario in one project and I have achieved it with windows power-shell script and Task Scheduled. Let me share you power-shell script for your reference.

    #Get 'n' number of files, you can set number as per your requirement
    $FileLimit = 5 
    #Pick-up location for files
    $PickupDirectory = Get-ChildItem -Path "\\Location\FilePick\"
    #Destination location for files
    $DropDirectory = "\\Location\FileDrop\"
    $Counter = 0
    foreach ($file in $PickupDirectory)
    {
        if ($Counter -ne $FileLimit)
        {
            $Destination = $DropDirectory+$file.Name
            Write-Host $file.FullName #Output file fullname to screen
            Write-Host $Destination   #Output Full Destination path to screen
            Move-Item $file.FullName -destination $Destination
            $Counter++
        }  
    }

    Please configure above script to Task scheduler task and set duration 5 min. This process will help you to pick 5 files after every 5 min.


    Thanks, If my reply is helpful please mark as answer or vote as helpful.

    • Proposed as answer by [Kamlesh Kumar] Sunday, August 23, 2015 6:27 PM
    • Marked as answer by Angie Xu Wednesday, August 26, 2015 6:55 AM
    Wednesday, August 19, 2015 5:35 PM
  • Thank you for your suggestions.

    Here the problem is, we cannot use the power script because we have many number of integrations which are going to Mainframes FTP location via BizTalk.

    The main issue here is, if we send 50 files via BizTalk, FTP location receives 55 files once. Next run the same 50 files are sent via BizTalk, FTP location receives 85 files and so on. There is no constant number which is going to FTP.

    We tried many options like, changing the Before Put value, changing the different combinations of Intervals and Redownload interval. Its not working.

    We suggested the clients about the power script. They are asking for singleton orchestration. But these flows are pass through. Is it the right option to use orchestration? please suggest.

    Thanks and Regards,

    Veena

    Monday, August 31, 2015 12:02 PM
  • Hi Veena,

    I'm not sure if you are still trying to resolve this requirement or not, but if so have you looked at the /n software FTP adapter?  This contains a property called "MaxFileCount" that can be used to limit the number of files downloaded in a single polling interval.  More information on on these adapters can be found on their website here:

    http://www.nsoftware.com/adapters/biztalk/


    Thanks,

    Adam

    Friday, October 30, 2015 3:49 PM