Is TPL dataflow for me ?


  • Hi,

    I am developing a middle tier application which basically receives messages from other applications on one end (END1), processes messages through various layers (each layer transmit the message to the next one, takes decisions based on the message (for routing to the correct next layer for example) and eventually increase the message with more data) and on the other end (END2) the transformed message is sent to another application. This application also receives messages from (END2) and does the same process but in reverse order to deliver message on the other end (END1). Some type of messages sent from an application are expecting a response to know if message was correctly processed, where some other type of messages are not expecting responses.

    It can be seen as a message adapter that receives message from one or the other end, works on messages through various layers, and finally transform the message to relay it to the other end in a format it can understand. Of course this middle tier app should be able to process multiple messages simultaneously without waiting for a message to be fully processed and transmitted to the other end before processing the next message.

    This middle tier application isn't compute bound at all, processing on messages is really light, but heavily I/O bound (receiving messages from HTTP or QUEUE and sending message via HTTP or TCP protocols ... querying and inserting into DB) , therefore there is no real need for parallelization at this time as there won't be no real gain from it.

    I have watched some TPL DataFlow videos and read some documentation about it, and really thrilled about it. It seems to correspond to what I am looking for, as messages can be passed around multiple layers, transformed, etc... and the naming "data flow" speaks to me as it is exactly what the application is. However from what I have seen/read the spotlight has really been put on parralelization aspect and this is not really what I require, but the message passing between layers is what interests me, and I really like this notion of blocks.

    I have started developing my application, relying heavily on new async/await constructs of C#5 due to the I/O bound aspect of this application, and for message passing between layers, basically each layer is trigerring an event to notify the other next layer (which is subscribed to the event) that a message has been received passing the message inside the event args. The event handlers are async event handlers, in order no to block the receiving of other messages during the treatment of one message. 

    But it doesn't seem really nice to me, I feel like this is kind of a code smell, I just don't like it. I would like to know if TPL Dataflow would be a good fit for my application considering I have no real need for parralelization.

    Hope my description / question is clear enough. Don't hesitate to ask for more details / clarification if needed.


    Friday, November 30, 2012 7:56 PM


All replies