none
Volume tracking RRS feed

  • Question

  • Hi all -

               I have an application which has the following components: receive location, map(at receive port) and a send port. The incoming files are distinguished by a character in the filename. This character is also a field in the incomign XML file.

    My questions are

    1. based on the character in the file name, can we get the number of files procesed during a day for each file type? Do I need to promote the property in the file to determine the file type?
    2. Can we get the total number of files successfully processed during the day(all files received at the port and sent)
    3. if the transformation fails, can we get the number of files failed during a day?

    If I can obtain the information above, what are the possible ways to do it? I am looking for direction from someone on which tools/features to use.

    Thank you.


    SRG
    Friday, January 28, 2011 12:53 PM

Answers

  • Hi there,

    an easy way to count things like that would be to use BAM. You could create a BAM activity and view (using Excel) which contains the file type, received timestamp and sent timestamp and whatever information (even from the message content) you like. Afterwards you deploy the BAM activity to the BAM database using the tool "bm.exe", which is located in the <BizTalk Installation directory>\Tracking

    Then you use the Tracking Profile Editor to create a tracking profile for the BAM activity, when the file is received at the port or send at the send port. After applying the tracking profile, you can use the BAM portal website (http://<Your BizTalk Server>/BAM) to look at the gathered data.

    In the end you can either use sql statements to aggregate the sums you need from the BAM Primary Import database or add dimensions and measures to your activity, which then uses SQL Analysis services to calculate these things for you.

    This would suffice your needs for 1. and 2.

    As answer to 3.: For failed mappings you need to enable "Routing for failed messages" on the receive port and write a small orchestration, which receives those messages. That orchestration could then send you an email alert or whatever you want, but more importantly you can create another BAM activity (using the steps described above) and write all information about those failed messages arriving at the orchestration to that activity.

    This should be enough to give you a start. BAM certainly requires deeper reading, so take a deep breath and dive into following MSDN section about BAM: http://msdn.microsoft.com/en-us/library/aa561326(v=bts.70).aspx

    Best regards,

    Leo


    Please mark it as Answer if this answers your question.
    Friday, January 28, 2011 2:03 PM
  • Thank you for confirming this for me. Really strange that this is missing I would say.To thread starter. Here is an example of a pipeline component you can use to log bam events from pipeline. You should expose relevant params, like name of the bam definition, if the event starts or ends etc. This should solve your needs.

    public
    
     IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
      {
       Stream originalStrm = null
    
    ;
       //BAM ActivityID
    
    
       string
    
     sActivityID = string
    
    .Empty;
       IBaseMessagePart bodyPart = inmsg.BodyPart;
       object
    
     sourceFileNameObj = inmsg.Context.Read("ReceivedFileName"
    
    , "http://schemas.microsoft.com/BizTalk/2003/file-properties"
    
    );
       string
    
     sourceFileName = string
    
    .Empty;
       if
    
     (sourceFileNameObj != null
    
    )
       {
        sourceFileName = sourceFileNameObj.ToString();
       }
       if
    
     (bodyPart != null
    
    )
       {
        // Get a new *REFERENCE* to the original input stream
    
    
        originalStrm = new
    
     SeekableReadOnlyStream(bodyPart.GetOriginalDataStream());
    
        if
    
     (originalStrm != null
    
    )
        {
         //declare BAM eventstream
    
    
         EventStream mes = null
    
    ;
         try
    
    
         {
          //using BTS InterchangeID as unique BAM ActivityID, because this is available in every step in bts
    
    
          sActivityID = (inmsg.Context.Read("InterchangeID"
    
    , "http://schemas.microsoft.com/BizTalk/2003/system-properties"
    
    )).ToString();
    
          if
    
     (sActivityID.Length != 0)
          {
           //init BAM eventstream
    
    
           mes = pc.GetEventStream();
           
           if
    
     (StartActivity)
           {
            //Creating new BAM Activity
    
    
            mes.BeginActivity(BAMActivity, sActivityID);
    
            //updating BAM with starting Log event
    
    
            if
    
     (BAMLogEvent != string
    
    .Empty)
            {
             mes.UpdateActivity(BAMActivity, sActivityID, BAMLogEvent, System.DateTime.Now);
            }
           }
           
           
           if
    
     (!string
    
    .IsNullOrEmpty(SourceFileName))
           {
            try
    
    
            {
             mes.UpdateActivity(BAMActivity, sActivityID, SourceFileName, sourceFileName);
            }
            catch
    
     (System.Exception ex)
            {
             string
    
     errorMsg = "Error when logging sourcefilename. "
    
     + ex.Message;
             if
    
     (ex.InnerException != null
    
    )
             {
              errorMsg += ex.InnerException.Message;
             }
             logger.Error(logProps, errorMsg);
            }
           }
           /*Will normally close an activity when Transmitting a message*/
    
           
           if
    
     (!StartActivity)
           {
            //updating BAM with closing Activity event
    
    
            mes.UpdateActivity(BAMActivity, sActivityID, BAMLogEvent, System.DateTime.Now);
            //Close BAM Activity
    
    
            mes.EndActivity(BAMActivity, sActivityID);
           }
           
           
          }
         }
         catch
    
     (Exception ex)
         {
          try
    
    
          {
           TraceUtils.Error("BAMLogPipelineComponent.Execute() failed."
    
    , ex);
           //throw ex;
    
    
           if
    
     (!StartActivity && mes != null
    
    )
           {
            mes.EndActivity(BAMActivity, sActivityID);
           }
          }
          catch
    
    
          {
          }
         }
         finally
    
    
         {
          //resetting stream
    
    
          if
    
     (originalStrm != null
    
     && originalStrm.CanSeek == true
    
    )
          {
           originalStrm.Seek(0, SeekOrigin.Begin);
          }
         }
        
        }
       }
       return
    
     inmsg;
     }
    

     

     

     

    Tuesday, March 8, 2011 9:18 AM

All replies

  • Hi there,

    an easy way to count things like that would be to use BAM. You could create a BAM activity and view (using Excel) which contains the file type, received timestamp and sent timestamp and whatever information (even from the message content) you like. Afterwards you deploy the BAM activity to the BAM database using the tool "bm.exe", which is located in the <BizTalk Installation directory>\Tracking

    Then you use the Tracking Profile Editor to create a tracking profile for the BAM activity, when the file is received at the port or send at the send port. After applying the tracking profile, you can use the BAM portal website (http://<Your BizTalk Server>/BAM) to look at the gathered data.

    In the end you can either use sql statements to aggregate the sums you need from the BAM Primary Import database or add dimensions and measures to your activity, which then uses SQL Analysis services to calculate these things for you.

    This would suffice your needs for 1. and 2.

    As answer to 3.: For failed mappings you need to enable "Routing for failed messages" on the receive port and write a small orchestration, which receives those messages. That orchestration could then send you an email alert or whatever you want, but more importantly you can create another BAM activity (using the steps described above) and write all information about those failed messages arriving at the orchestration to that activity.

    This should be enough to give you a start. BAM certainly requires deeper reading, so take a deep breath and dive into following MSDN section about BAM: http://msdn.microsoft.com/en-us/library/aa561326(v=bts.70).aspx

    Best regards,

    Leo


    Please mark it as Answer if this answers your question.
    Friday, January 28, 2011 2:03 PM
  • Leo,

             Thanks so much for the prompt and detailed answer. This is helpful.

    Thanks again.


    SRG
    Friday, January 28, 2011 2:08 PM
  • I have a question about this. I have been using BAM in various scenarious, but I have always used the BAM API, and tracked via OrchestrationEventStream in an orchestration and even via pipelines so the Tracking profile editor is new to me.

     

    Now I have a scenario very much like the thread starter here, but I can't figure out how to use the Tracking Profile Editor in this scenario, Meaning, it looks to me like I need to load an orchestration to tie into my milestones(Message received, message sendt).

     

    Can somebody excplain how to use the Tracking Profile Editor in a pure message routing scenario where no orchestration is involved?

    Monday, March 7, 2011 2:26 PM
  • Leo: TPE cannot work with ports only with orchestrations. There is no orch in this solution.

    I think there should be the additional pipeline component to fire the BAM tracking events.


    Leonid Ganeline [BizTalk MVP] BizTalkien: Suspend shape and Convoy
    Tuesday, March 8, 2011 6:01 AM
    Moderator
  • Hi,

    you can use the TPE in a messaging only solution. We use it in our customer solutions.

    1. You need to select message context or content in the DropDownBox in TPE and drag the required information to the node on the left hand side.
    2. Right-click the new entry in the tree view and select Port Mapping. There you can choose on which port you want to gather this information.

    As ActivityID you could use the InterchangeID, which stays the same in a messaging only solution.

    Regards,
    Leo


    Please mark it as Answer if this answers your question.
    Tuesday, March 8, 2011 9:09 AM
  • Thank you for confirming this for me. Really strange that this is missing I would say.To thread starter. Here is an example of a pipeline component you can use to log bam events from pipeline. You should expose relevant params, like name of the bam definition, if the event starts or ends etc. This should solve your needs.

    public
    
     IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
      {
       Stream originalStrm = null
    
    ;
       //BAM ActivityID
    
    
       string
    
     sActivityID = string
    
    .Empty;
       IBaseMessagePart bodyPart = inmsg.BodyPart;
       object
    
     sourceFileNameObj = inmsg.Context.Read("ReceivedFileName"
    
    , "http://schemas.microsoft.com/BizTalk/2003/file-properties"
    
    );
       string
    
     sourceFileName = string
    
    .Empty;
       if
    
     (sourceFileNameObj != null
    
    )
       {
        sourceFileName = sourceFileNameObj.ToString();
       }
       if
    
     (bodyPart != null
    
    )
       {
        // Get a new *REFERENCE* to the original input stream
    
    
        originalStrm = new
    
     SeekableReadOnlyStream(bodyPart.GetOriginalDataStream());
    
        if
    
     (originalStrm != null
    
    )
        {
         //declare BAM eventstream
    
    
         EventStream mes = null
    
    ;
         try
    
    
         {
          //using BTS InterchangeID as unique BAM ActivityID, because this is available in every step in bts
    
    
          sActivityID = (inmsg.Context.Read("InterchangeID"
    
    , "http://schemas.microsoft.com/BizTalk/2003/system-properties"
    
    )).ToString();
    
          if
    
     (sActivityID.Length != 0)
          {
           //init BAM eventstream
    
    
           mes = pc.GetEventStream();
           
           if
    
     (StartActivity)
           {
            //Creating new BAM Activity
    
    
            mes.BeginActivity(BAMActivity, sActivityID);
    
            //updating BAM with starting Log event
    
    
            if
    
     (BAMLogEvent != string
    
    .Empty)
            {
             mes.UpdateActivity(BAMActivity, sActivityID, BAMLogEvent, System.DateTime.Now);
            }
           }
           
           
           if
    
     (!string
    
    .IsNullOrEmpty(SourceFileName))
           {
            try
    
    
            {
             mes.UpdateActivity(BAMActivity, sActivityID, SourceFileName, sourceFileName);
            }
            catch
    
     (System.Exception ex)
            {
             string
    
     errorMsg = "Error when logging sourcefilename. "
    
     + ex.Message;
             if
    
     (ex.InnerException != null
    
    )
             {
              errorMsg += ex.InnerException.Message;
             }
             logger.Error(logProps, errorMsg);
            }
           }
           /*Will normally close an activity when Transmitting a message*/
    
           
           if
    
     (!StartActivity)
           {
            //updating BAM with closing Activity event
    
    
            mes.UpdateActivity(BAMActivity, sActivityID, BAMLogEvent, System.DateTime.Now);
            //Close BAM Activity
    
    
            mes.EndActivity(BAMActivity, sActivityID);
           }
           
           
          }
         }
         catch
    
     (Exception ex)
         {
          try
    
    
          {
           TraceUtils.Error("BAMLogPipelineComponent.Execute() failed."
    
    , ex);
           //throw ex;
    
    
           if
    
     (!StartActivity && mes != null
    
    )
           {
            mes.EndActivity(BAMActivity, sActivityID);
           }
          }
          catch
    
    
          {
          }
         }
         finally
    
    
         {
          //resetting stream
    
    
          if
    
     (originalStrm != null
    
     && originalStrm.CanSeek == true
    
    )
          {
           originalStrm.Seek(0, SeekOrigin.Begin);
          }
         }
        
        }
       }
       return
    
     inmsg;
     }
    

     

     

     

    Tuesday, March 8, 2011 9:18 AM
  • Hi,

    you can use the TPE in a messaging only solution. We use it in our customer solutions.

    1. You need to select message context or content in the DropDownBox in TPE and drag the required information to the node on the left hand side.
    2. Right-click the new entry in the tree view and select Port Mapping. There you can choose on which port you want to gather this information.

    As ActivityID you could use the InterchangeID, which stays the same in a messaging only solution.

    Regards,
    Leo


    Please mark it as Answer if this answers your question.

    Thanks, Leo!
    Leonid Ganeline [BizTalk MVP] BizTalkien: Suspend shape and Convoy
    Tuesday, March 8, 2011 2:56 PM
    Moderator
  • I am looking for a solution in messaging only applications. I will try this option.

    thank you


    SRG
    Tuesday, March 8, 2011 3:34 PM