none
Deserialize TCP data inside UDSO

    Question

  • Hi,

    I am developing SI application where read data from TCP and apply query on it. My TCP data has much complex structure.

    I have question on this:

    1) Is it good practise that, I pass all TCP raw data stream to streamInsight and deserialize that stream using USDO?

    Or

    Should I deserialze it and then send to streamInsight engine?

    Thanks in advance.

    Friday, April 12, 2013 5:37 AM

All replies

  • In general, it is better to deserialize in your input adapter and create the payload before sending to the engine.

    That said, there are also cases where that simply won't work. For example, if your TCP data has a complex structure and fields that may differ based on a header value, you won't be able to reasonably to that in an input adapter/source. For example, let's say that your TCP data looks like the following:

    Header
    Timestamp
    Source
    Type
    Payload

    If Type="A" Then Payload is:
    Lat
    Long
    Temp

    If Type="B" Then Payload is:
    WindSpeed
    Direction
    Lat
    Long

    In this case, you can't create a single event type for the event. Now, you could do multiple input adapters for each type but that wouldn't necessarily be very efficient. So, you can create a single "inbound" type that has the header and the payload as a byte array and then a UDSO to further deserialize based on the Payload type.


    DevBiker (aka J Sawyer)
    Microsoft MVP - Sql Server (StreamInsight)


    Ruminations of J.net


    If I answered your question, please mark as answer.
    If my post was helpful, please mark as helpful.

    Friday, April 12, 2013 4:06 PM
  • Thanks for response,

    How I did, I am using WCF source and sink for my application. I deserialize TCP data upto some extend so that i can recognise the type and payload then rest i am deserializing it using UDSO.

    Is this good design to have WCF source and sink in this scenario?

    or

    When should we use WCF source and Sink?

    After process data through I need to show it in different application like (window app, web application).

    Tuesday, April 16, 2013 9:49 AM
  • OK, it sounds like your approach - using the UDSO as a "second-stage" deserialization - is the best one for your situation.

    Is it good to have WCF in this scenario? As always, it depends. WCF is pretty awesome in that it's easy to hook systems together over the wire. But it also has more overhead than some other methods - like raw TCP sockets, for example. Some of this can be overcome by using different bindings but not all. The question, really, is whether it'll scale to the event rate that you need (in most cases it will) and if it's appropriate for your source or target. To send to a windows app, WCF Duplex will usually work pretty well, be fairly straightforward and easy and sufficiently scale. Web apps, however, are more difficult to "push" to ... you might want to look at Web Sockets (for HTML 5) or Silverlight WCF Polling Duplex.


    DevBiker (aka J Sawyer)
    Microsoft MVP - Sql Server (StreamInsight)


    Ruminations of J.net


    If I answered your question, please mark as answer.
    If my post was helpful, please mark as helpful.

    Tuesday, April 16, 2013 4:04 PM