locked
Query works in Test mode but fails with transient failure RRS feed

  • Question

  • Hi,

    I have a query that works well while testing with input json data. However, when the job is started the query fails with transient failure.

    I have two eventhub input sources input1 and input2. I also have a reference data source in a blob, ref.

    The query is 

    WITH nameone AS(
    
    SELECT ref.field1 as field1,ref.field2 as field2,MIN(input1.datetimestamp) AS datetimestamp,avg(input1.value) as value FROM input1 TIMESTAMP BY datetimestamp JOIN ref ON ref.field1 = input1.field1 GROUP BY ref.field1,tumblingwindow(second,30)
    
    ), nametwo AS(
    
    SELECT ref.field1 as field1,ref.field2 as field2,MIN(input2.datetimestamp) AS datetimestamp,avg(input2.value) as value FROM input2 TIMESTAMP BY datetimestamp JOIN ref ON ref.field1 = input2.field1 GROUP BY ref.field1,tumblingwindow(second,30)
    
    )
    
    SELECT field1, field2,datetimestamp,value from nameone
    
    UNION
    
    SELECT field1, field2,datetimestamp,value from nametwo

    Thanks

    Friday, March 6, 2015 5:53 PM

Answers

  • I solved the problem. 

    The issue was in conversion of incoming data to write into an SQL db.

    Query failed to convert 

     System.Int64 (incoming data format) to  System.Double ( SQL column data format)

    I solved it by changing SQL column format to int, but I think this casting should be done automatically by the query. I can also CAST the data I guess but sometimes using CAST on everything is kind of a pain.

    Another issue I faced was the failure messages. I checked the logs for last 12 hours. Sometimes the failure will point to the Writing Error (conversion of int to double) but sometimes the failure will simply be transient failure. So it was difficult to pinpoint the error exactly.

    Thanks anyways.


    • Marked as answer by Zafar Abbas Thursday, March 19, 2015 6:50 PM
    Thursday, March 19, 2015 4:45 PM

All replies

  • Hello, Could you please share: Your job name The details of the message you get in operations log We will take a look at our logs. Thanks

    Zafar Abbas


    • Edited by Zafar Abbas Saturday, March 7, 2015 7:12 PM
    Saturday, March 7, 2015 7:12 PM
  • How can I share that information with you? Is there any way I can send a personal message to you or the support?

    Thanks

    Monday, March 9, 2015 9:18 PM
  • You can send the details to azstream@microsoft.com

    Thanks


    Zafar Abbas

    Tuesday, March 10, 2015 5:35 AM
  • Thanks

    I have emailed  azstream@microsoft.com

    Tuesday, March 10, 2015 4:15 PM
  • Thanks for the details, we have identified an issue in our backend and working on a resolution. We will keep you posted as soon as we can a fix rolled out.

    Thanks


    Zafar Abbas

    Wednesday, March 11, 2015 6:26 PM
  • The query works well now. 

    Thanks for your help

    Tuesday, March 17, 2015 3:33 AM
  • Actually it still fails in some cases. Even simpler query which was working before is now failing!
    • Edited by ApSam Thursday, March 19, 2015 3:10 PM
    Wednesday, March 18, 2015 6:43 PM
  • Could you please send the failing jobs and queries to azstream@microsoft.com and we will take a look at it right away.

    Thanks!


    Zafar Abbas

    Thursday, March 19, 2015 4:21 PM
  • I solved the problem. 

    The issue was in conversion of incoming data to write into an SQL db.

    Query failed to convert 

     System.Int64 (incoming data format) to  System.Double ( SQL column data format)

    I solved it by changing SQL column format to int, but I think this casting should be done automatically by the query. I can also CAST the data I guess but sometimes using CAST on everything is kind of a pain.

    Another issue I faced was the failure messages. I checked the logs for last 12 hours. Sometimes the failure will point to the Writing Error (conversion of int to double) but sometimes the failure will simply be transient failure. So it was difficult to pinpoint the error exactly.

    Thanks anyways.


    • Marked as answer by Zafar Abbas Thursday, March 19, 2015 6:50 PM
    Thursday, March 19, 2015 4:45 PM