none
EDI Output Batching for large volume of records RRS feed

  • Question

  • For a TPA, we have a requirement to generate X12 EDI output file. We have an EDI message only scenario, and need to generate Eligibility Roster for 200,000 eligibility records that need to be processed through BizTalk 2009 and 1 EDI output file needs to be created for all the 200,000 records.

    What would be the best approach to realize this requirement?

    Environment Details: BizTalk 2009 Server Farm (2 servers), Windows 2003 R2 64 bit, SQL server 2005 clustered.

    Let me know if any additional information may be required.

    Friday, March 9, 2012 3:27 AM

All replies

  • I don't recall the streaming model used in the EDI assembler, but it's probably similar to the flat file assembler with overhead for EDI interchange processing (enveloping).  The flat file assembler is capable of serializing a substantial number of flat files.  This is mainly because although the size of the individual messages might be very large, the internal memory footprint only holds stream pointers + the overhead of the individual stream being serialized at any one time.

    Have you tried prototyping assembling a large number of EDI messages on your development computer?  This should provide a good benchmark to assist in making your decision.

    Hopefully, you will be pleasantly suprised.


    David Downing... If this answers your question, please Mark as the Answer. If this post is helpful, please vote as helpful.

    Friday, March 9, 2012 2:29 PM
  • Not yet, but I will start working on it from this weekend.

    I just wanted to pick brains of the folks who might have already encountered this so that I am not spending time on approaches which have been already tried.

    I will post an update once I am done with the POC.

    Friday, March 9, 2012 4:32 PM
  • Without being able to prototype the type of load (different types of messages have substantially different processing footprints), it's extremely difficult to provide a generic scale out response.  We went to the Microsoft performance lab before our launch and perf tested for more than a month... we were able to duplicate our production environment in the lab.  This was extremely value in tuning, SQL, the network, BizTalk, ...

    David Downing... If this answers your question, please Mark as the Answer. If this post is helpful, please vote as helpful.

    Friday, March 9, 2012 4:41 PM