locked
Dispatch queries return unusable connectedServiceResult / childApp results RRS feed

  • Question

  • Hi,

    starting this week we've experienced very strange results from our Dispatch applications in the westeurope region.

    We've got a LUIS application and a dispatcher which contains it. Both have only 1 version and are published both on staging and production slots. 

    When directly sending a query to the LUIS application, we get proper results. The correct intent with scores above 0.8.

    Unfortunately, when sending the same query to the dispatch application, we only get the correct root intent (which points to the correct LUIS app at least). However, it returns some really wild guesses for the intent of the LUIS app. The scores are at most at 0.2. More often they're below 0.1.

    Is there any known issue or something I could be missing? Older dispatch applications created with the same script work fine and newer test apps I've just created have the same issues. I'm afraid older apps might also become broken when retraining them, so I'd not like to do this... ;)

    I'd appreciate any ideas or direct help from MS.

    Thanks and kind regards
    Sebastian

    Friday, March 20, 2020 3:40 PM

All replies

  • Hi Sebastian,

    It would be great if you can let us know the version of dispatch tool that you are currently facing issues with. We are checking with our internal team for any recent updates to LUIS or the tool that could be causing this. In the meanwhile if you could share us examples of this behavior it would help us share the same to the team for deeper analysis of this issue.

    -Rohit

    Monday, March 23, 2020 10:33 AM
  • Hello Sebastian,

    Based on feedback provided from our product team could you please export your parent/child app JSON and check if the following match in both the files?

    1) Parent has different tokenizer version than child
    2) Parent has different normalization settings than child

    If the above mismatch occurs then the scores seen are possibly by design and not an issue from the dispatch tool.

    Also the dispatch tool was not updated in the last month and the tool does not control the scores that are returned from the app. 

    -Rohit

    Monday, March 23, 2020 4:28 PM
  • Hi Rohit,

    thanks a lot for your reply containing some ideas from the team already!

    1) the tokenizer version is 1.0.2 for both.

    2) I guess the "normalization settings" are the "settings" field in the JSON, as I could not find any search result for "normaliz" in the whole JSON export? The differences are very small:

    2.1) In the dispatch app, "NormalizeDiacritics" is set to "true" and I have no idea how to set/change this in the child app.

    2.2) However, since you've mentioned the settings, I've now toggled the "Use non-deterministic training" toggle in the LUIS preview portal. I've set it to "off" in both apps, trained and published both. Then I've turned it "on" again (which should be the recommended way), trained and published and now it seems to work properly

    When I posted the issue originally, "UseAllTrainingData" was not set in the "settings" object. Now it's explicitly set to "false" for both in their exported JSON representation. I'm not sure whether this did the trick or whether it just was a coincidence.

    Thanks for pointing out how this works and I'm glad that the team's ideas already lead me to a solution! <3

    Tuesday, March 24, 2020 7:35 AM
  • Okay, unfortunately it was working for some samples. But meanwhile it turns out that other ones, where the exact query phrase is part of the training set for an intent, it still goes very wrong.

    Is there anything more to check besides the version and the settings?

    Tuesday, March 24, 2020 9:53 AM
  • Hello Sebastian,

    Is it possible to provide information regarding these queries through the LUIS portal from General Inquiry through User voice(Simley on the top right) with a link to this thread? This will help our service team to lookup the exact queries and provide a more accurate response. Based on some recent issues reported the tokenizer and Normalization settings were some probable causes the team hinted at. If you are seeing further issues like this we would recommend to pass the information through User voice of LUIS portal.

    -Rohit

    Thursday, March 26, 2020 9:21 AM
  • Thanks Rohit! I just followed your recommendation to send a UserVoice inquiry.
    Thursday, March 26, 2020 10:30 AM