locked
Consuming simple string input RRS feed

  • Question

  • I'm getting data from sensors to my IoT Hub in the below format:

    $GPRP,083B6F936228,F0604BD6CAD7,-90,1EFF060001092002111EF6919DE8B42BA4849F0D1947151F4737314CC35F86

    The BLE-WiFi gateway gets the data from the sensors and sends to IoT Hub via MQTT. But in my Streaming analytics Job, I'm not able to accept this data as CSV. Keeps on getting serialization error:

    Could not deserialize the input event(s) from resource 'Partition: [2], Offset: [12897374080], SequenceNumber: [254913]' as Csv. Some possible reasons: 1) Malformed events 2) Input source configured with incorrect serialization format

    May be the error is due to missing headers which makes the data an invalid CSV; but this is what I get from the sensors!

    Is there anyway I could tell the streaming analytics job to just consider this as a simple string? I can do the conversions using JavaScript UDF's in the stream analytics.


    • Edited by Kiran U Tuesday, October 16, 2018 1:04 PM
    Tuesday, October 16, 2018 1:00 PM

All replies

  • Hi Kiran,

    Unfortunately, at this time, ASA requires CSV with headers. UDF function can't be used for this purpose.

    If you run ASA on IoT Edge, it is possible to use our custom deserializer feature announced at Ignite (available in private preview).

    Thanks,

    JS

    • Proposed as answer by Jason_J (Azure) Wednesday, October 17, 2018 6:01 PM
    Tuesday, October 16, 2018 3:41 PM
  • Hi Jean,

    Thank you for your response. I don't think I can go with IoT Edge as the device does not give me much control over sending the data. All I can do is configure the MQTT setup in the device configuration.

    However, the "at this time" in the response gives me some hope that in future ASA might support CSV without headers. May be an extra configuration in the ASA to specify the CSV headers..

    May be I'm wrong, but isn't this a basic scenario where the IoT devices sending data in such a format? If not header-less CSV support, support for accepting as a plain string would have suffice my requirement. I could have done the comma split and other stuff in the UDF. 

    Thanks,

    Kiran

    Monday, October 22, 2018 5:16 AM