locked
[UWP] Windows.Media.SpeechSynthesis Feedback RRS feed

  • Question

  • Hi Everyone

    I am looking for a way to get some sort of information back from the MediaElement or SpeechSynthesis that I can use to make mouth shapes on a talking head.

    •  I have looked for Viseme events (not available in UWP).
    • I have tried looking for markers when playing the speech audio. (not included in UWP).
    • I have tried to find current audio volume from the media element which would be like an audio meter (not available in UWP).

    Does anyone have any other idea how I could do it?

    string text = "The quick brown fox jumped over the lazy dog";
    
    SpeechSynthesisStream synthesisStream = await synthesizer.SynthesizeTextToStreamAsync(text);
    
    // Usually embedded in Xaml
    MediaElement media = new MediaElement();
    media.AutoPlay = true;
    media.SetSource(synthesisStream, synthesisStream.ContentType);
    media.Play();

    thanks


    Saturday, April 1, 2017 12:46 AM

Answers

  • Markers are supported if you provide them in your SSML (see https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesisstream#properties_ 's Markers property), and you should be able to get the audio levels in the MediaElement by adding an MFT (AddAudioEffect) to filter the stream as it goes through and read the raw data.

    The Creator's Update has some new SpeechSynthesisOptions to provide word and sentence boundary metadata that may help you.

    • Marked as answer by hypodyne1 Thursday, April 6, 2017 11:38 AM
    Saturday, April 1, 2017 1:14 AM

All replies

  • Markers are supported if you provide them in your SSML (see https://docs.microsoft.com/en-us/uwp/api/windows.media.speechsynthesis.speechsynthesisstream#properties_ 's Markers property), and you should be able to get the audio levels in the MediaElement by adding an MFT (AddAudioEffect) to filter the stream as it goes through and read the raw data.

    The Creator's Update has some new SpeechSynthesisOptions to provide word and sentence boundary metadata that may help you.

    • Marked as answer by hypodyne1 Thursday, April 6, 2017 11:38 AM
    Saturday, April 1, 2017 1:14 AM
  • Hi Rob

    Thanks for the answer.

    The markers might work. Going to try that for now.

    Anyone else have any ideas?

    Monday, April 3, 2017 7:34 PM