none
How process the empty CSV file with header

    Question

  • Hi,

    In our project every day we receive CSV files (8 files with multiple rows in it and with headers). Receive pipeline recognizes files with headers, and send them to appropriate branch to process them, and after processing received file it will add +1 to count how many files were finished the process.

    But in some cases, the received file can have a header with an empty body. In this case, Biztalk brings and delete those files without counting them and processing them. I don't need to process that file, I just need to count +1 for that file too as received and processed file.

    Is there any easy way to count all received files and even those files which received with an empty body (but with headers)? I know that I can reach this with a custom pipeline, but it's a bit harder to me write custom pipeline. Any suggestions?

    Thanks in advance.
    Tuesday, February 12, 2019 11:32 AM

All replies

  • Can you explain where the solution is currently doing that count? Are you using an Orchestration?

    The BizTalk file adapter would delete any files of 0 length, as that is not your situation as it has a header, so then there must already have some logic somewhere that is discarding the message if it only has a header.  So the logic to increase the count should occur before that bit of logic.


    Tuesday, February 12, 2019 7:41 PM
  • Thanks for your response,

    Yes I am using Orchestration and there I count rows of the file with xpath and after process, I add +1 to a variable which counts how many files were received and processed.
    But in this case, if the file will be empty it's ok if it will count that file as processed and give +1 to processed files count before Orchestration.
    And how can I give +1 to the variable if it is empty? Only with a custom pipeline, or there is another way not hard way?

    Thanks.

    Wednesday, February 13, 2019 5:38 AM
  • So what is stopping you adding +1 to the variable even if the xpath count says it is 0 lines?

    Can you give us an overview of your Orchestration?

    Wednesday, February 13, 2019 8:20 PM
  • The problem is that message is not going to Orchestration where I am counting rows count with XPath or adding +1 to the variable (if it has only a header). All the logic is in orchestration.

    There may be 2 ways to add +1 I guess. 

    1) File (empty body with header) somehow BizTalk brings to Orchestration where I add +1 to my variable.

    2) Add +1 somehow before the file (empty body with header) will be deleted by BizTalk.



    P.S.  Orchestration logic is very big, but in this case, I need to focus only on adding +1 when one of the received files has an empty body. In short every day we receive 8 different files with multiple rows. And in Orchestration, we count which file has completed the process, after that, every time will add +1 to a variable to notify that file was stored in CRM. After all files process will be finished (it means the variable count will be 8), it has to do other operations with already stored records in CRM. Logic is very similar to Parallel Action, but with my files, Parallel Action can't work how we want.

    So basically one of the mentioned two points will solve my problem.

    Thanks.


    Thursday, February 14, 2019 5:35 AM
  • The BizTalk file adapter would delete any files of 0 length, as that is not your situation as it has a header, so then there must already have some logic somewhere that is discarding the message if it only has a header.  So the logic to increase the count should occur before that bit of logic.

    It seems that BizTalk adapter would delete not only files of 0 lengths, but it would also delete a file which has header without a body. Because when I put that kind of file it disappears. It can't reach to orchestration, even when my receive pipelines  'Preserve header' is set to 'True'.
    Thursday, February 14, 2019 10:35 AM