none
How to use a queue in a windows service RRS feed

  • Question

  • Hi,
    I'm running a windows service which collect string messages from another service. Once I have the message I need to write these messages to a file. Sometimes I my service get a huge amount of messages and fails to write all of them to the disk.

    My idea is to queue the messages as and when they receive an then use another process to read the queue and write the messages to the disk.

    Is this the best way of handling this and is it better to use the message queue or simply use a queue object to do this. Any advice is appreciated.

    Thanks
    VJ
    Wednesday, June 11, 2008 8:30 AM

Answers

  • VJ,

    MSMQ is the most reliable communication mechanism. All messages will be preserved even if your Windows Service is down.

    I believe MSMQ is the best choice in your case.
    Vitaliy Liptchinsky
    • Marked as answer by Vipstercular Wednesday, June 11, 2008 10:18 AM
    Wednesday, June 11, 2008 10:07 AM
  • I am confused why your service fails to write the messages to disk. The easiest thing would be to fix the error and you are fine. MSMQ is not a solution because it will write the message to disk and you will read from the other process the message from MSMQ hence also from disk only to write it back to your hard drive. The performance of this solution will be suboptimal to say the least. If you receive a message do not return from the log call until you did write it to disk and flush the stream after the write operation. That ensures reliability at the cost of a few percent throughput performance.
    Perhaps you should consider to use a logging framework like the Microsoft Enteprise Library Logging Application Block, NLog, Log4Net, ....
    http://msdn.microsoft.com/en-us/library/cc467894.aspx
    http://www.codeproject.com/KB/architecture/GetLoggingWithEntLib.aspx


    Yours,
       Alois Kraus
    • Marked as answer by Vipstercular Thursday, June 12, 2008 11:53 AM
    Wednesday, June 11, 2008 11:29 AM

All replies

  • Hi VJ,

    You can use standard Microsoft Message Queue instead of creation of your own. If this is one-way communication (service only receives messages, but does not send response) and communication performance is not so important then I believ MSMQ would be the best choice here.

    Vitaliy Liptchinsky
    Wednesday, June 11, 2008 8:43 AM
  • Hi,
    Thanks for the reply. This is one way communication but I need to make sure that the service will not lose any messages.  Performance wise , if the queue can enque all the messages from the service that's all I need. There is no real time contraint on writing the message to the disk.

    Do you think MSMQ would suit the situation.

    Thanks.
    VJ
    Wednesday, June 11, 2008 9:58 AM
  • VJ,

    MSMQ is the most reliable communication mechanism. All messages will be preserved even if your Windows Service is down.

    I believe MSMQ is the best choice in your case.
    Vitaliy Liptchinsky
    • Marked as answer by Vipstercular Wednesday, June 11, 2008 10:18 AM
    Wednesday, June 11, 2008 10:07 AM
  • Much appreciate your advice. I'm going to use it.

    Thanks again,
    VJ
    Wednesday, June 11, 2008 10:19 AM
  • I am confused why your service fails to write the messages to disk. The easiest thing would be to fix the error and you are fine. MSMQ is not a solution because it will write the message to disk and you will read from the other process the message from MSMQ hence also from disk only to write it back to your hard drive. The performance of this solution will be suboptimal to say the least. If you receive a message do not return from the log call until you did write it to disk and flush the stream after the write operation. That ensures reliability at the cost of a few percent throughput performance.
    Perhaps you should consider to use a logging framework like the Microsoft Enteprise Library Logging Application Block, NLog, Log4Net, ....
    http://msdn.microsoft.com/en-us/library/cc467894.aspx
    http://www.codeproject.com/KB/architecture/GetLoggingWithEntLib.aspx


    Yours,
       Alois Kraus
    • Marked as answer by Vipstercular Thursday, June 12, 2008 11:53 AM
    Wednesday, June 11, 2008 11:29 AM
  • Alois said:

    I am confused why your service fails to write the messages to disk. The easiest thing would be to fix the error and you are fine. MSMQ is not a solution because it will write the message to disk and you will read from the other process the message from MSMQ hence also from disk only to write it back to your hard drive.

    You can make MSMQ write to another physical machine. I believe problem here is not concerned with disk IO operations, but with overload of service. Am I right, VJ?


    Vitaliy Liptchinsky
    Wednesday, June 11, 2008 12:47 PM
  • Hi,
    My service should read a messages from clients and write them to a disk as well as insert into a database too . Since writing to a disk is a slower operation I like to avoid the effect of delay on service caused by writing. My idea is to queue messages to a queue (MSMQ or internal queue)  as and when the messages arrive, and read it from that queue to write to the disk.

    Hope this explains my intention. Is there a better way to achieve this.  I want my service  not to be slow and specially not to miss any messages.

    many thanks.
    VJ
    Thursday, June 12, 2008 11:08 AM
  • vije said:

    Hi,
    My service should read a messages from clients and write them to a disk as well as insert into a database too . Since writing to a disk is a slower operation I like to avoid the effect of delay on service caused by writing. My idea is to queue messages to a queue (MSMQ or internal queue)  as and when the messages arrive, and read it from that queue to write to the disk.

    Hope this explains my intention. Is there a better way to achieve this.  I want my service  not to be slow and specially not to miss any messages.

    many thanks.
    VJ

    as you proposed on the previous post I tried flushing technique.

      using (StreamWriter sw = File.AppendText(SaveFilePath))
                    {
                        sw.WriteLine(FIXMessage + " | " + DateTime.Now.ToString() + " | " + sessionID.ToString() );
                        sw.Flush(); sw.Close();
                    }
                    return true;

    I flooded the service with client requests. Send out 10,000 messages wit in 5Min 31 sec. My method manage to write all the messages to the disk with 0% loss.

    Thanks for the advice. Alios - thanks for your advice and Vitaliy- thanks for introducing the MSMQ. I believe that MSMQ has a lot of potential when it comes to system integrations.

    VJ


    • Edited by Vipstercular Thursday, June 12, 2008 12:00 PM Updated code
    Thursday, June 12, 2008 11:58 AM
  • Hi Guys,

    I implemented my whole routine  ie. writing to the disk and inserting to the database. The database insertion is a little time consuming process and therefor my service is not responsive enough to the clients. Clients are timeoutting  

    What could be the best way to make the service more responsive ?

    My idea  : Write the messages to a MSMQ and run a different service to read from the MSMQ.

    Ideally I would like to have the whole process with in the same service. Can I use multi threading or Asynchronous programming to achieve this ?  The insertion to the database has to be done sequentially. in otherwords the message insertion is not independent to each other.

    Any advice is much appreciated.

    Many thanks,
    VJ

    Friday, June 13, 2008 2:15 PM