none
Live streaming on Windows IoT Core RRS feed

  • Question

  • Hello guys!

    I am currently trying to build a solution that uses a raspberry pi to make a home security system that is connected to a webcam and would like to stream the content if another client (for example an android application) wants to. I am trying to use azure media services as a proof of concept, but i dont seem to know if it is workable.. can you guys help me elaborate? i tried using the sdk for azure media services (nuget package), but if i am correct, it is incompatible with the uap project im working on for the raspberry pi
    Thursday, March 22, 2018 4:29 AM

Answers

  • Not sure exactly what you need here, but I would not recommend using the Azure Media services .NET SDK on the PI.  It is not .NET CORE.  

    If you want to use Media Services APIs from the PI, then look at using the raw REST APIs. We have a POSTMAN collection up here.

    In terms of streaming, you will need to send the content as RTMP or SMOOTH.  
    If using FFMPEG, I would recommend using Smooth format.

    Looking at my own old notes from Raspberry Pi, i think I got this to work once. You will need to play with the bitrate and scaling to optimize the CPU.  

    Note that I show in this example how to use the Hardware OMX optimizations. 

    To Enable hardware acceleration options:

    • add “–enable-omx –enable-omx-rpi” to ./configure options
    1. use encoder option “-c:v h264_omx”

    <cite style="margin:0in;margin-font-family:Calibri;font-size:9.0pt;color:#595959;">      From <https://ubuntu-mate.community/t/hardware-h264-video-encoding-with-libav-openmax-il/4997/11> </cite>

    SMOOTH STREAMING BROADCAST

    ffmpeg  -i /dev/video1 -pix_fmt yuv420p -f ismv -movflags isml+frag_keyframe  -video_track_timescale 10000000 -frag_duration 2000000 -framerate 30 -r 30  -c:v h264_omx -preset ultrafast -map 0:v:0  -b:v:0 2000k -minrate:v:0 2000k -maxrate:v:0 2000k -bufsize 2500k  -s:v:0 640x360  -map 0:v:0  -b:v:1 500k -minrate:v:1 500k -maxrate:v:1 500k -s:v:1 480x360 -g 60 -keyint_min 60 -sc_threshold 0  -c:a libfaac -ab 48k  -map 0:a? -threads 0 "http://XXXXXXXX.channel.mediaservices.windows.net/ingest.isml/Streams(video)"

     

    EXPLANATION OF WHAT IS GOING ON ABOVE ON IN A TYPICAL FFMPEG COMMAND LINE.

     

    ffmpeg

     

    -re     **READ INPUT AT NATIVE FRAMERATE

    -stream_loop -1  **LOOP INFINITE

    -i C:\Video\tears_of_steel_1080p.mov   **INPUT FILE IS THIS MOV FILE - USE the Device flags later once you test

    -movflags isml+frag_keyframe  **OUTPUT IS SMOOTH STREAMING THIS SETS THE FLAGS

    -f ismv  **OUTPUT ISMV SMOOTH

    -threads 0  ** SETS THE THREAD COUNT TO USE FOR ALL STREAMS. YOU CAN USE A STREAM SPECIFIC COUNT AS WELL

    -c:a aac  ** SET TO AAC CODEC

    -ac 2   ** SET THE OUTPUT TO STEREO

    -b:a 64k ** SET THE BITRATE FOR THE AUDIO

     -c:v libx264  ** SET THE VIDEO CODEC

    -preset fast ** USE THE FAST PRESET FOR X246, play around with other options.

     -profile:v main **USE THE MAIN PROFILE

    -g 48 ** GOP SIZE IS 48 frames

     -keyint_min 48 ** KEY INTERVAL IS SET TO 48 FRAMES

    -sc_threshold 0  ** NOT SURE WHY THIS IS SET. 

    -map 0:v   ** MAP THE FIRST VIDEO TRACK OF THE FIRST INPUT FILE

    -b:v:0 5000k   **SET THE OUTPUT TRACK 0 BITRATE

    -minrate:v:0 5000k  ** SET OUTPUT TRACK 0 MIN RATE TO SIMULATE CBR

    -maxrate:v:0 5000k  ** SET OUTPUT TRACK 0 MAX RATE TO SIMULATE CBR

    -s:v:0 1920x1080  **SCALE THE OUTPUT OF TRACK 0 to 1920x1080.

    -map 0:v  ** MAP THE FIRST VIDEO TRACK OF THE FIRST INPUT FILE

    -b:v:1 3000k ** SET THE OUTPUT TRACK 1 BITRATE TO 3Mbps

    -minrate:v:1 3000k -maxrate:v:1 3000k  ** SET THE MIN AND MAX RATE TO SIMULATE CBR OUTPU

    -s:v:1 1280x720  ** SCALE THE OUTPUT OF TRACK 1 to 1280x720

    -map 0:v -b:v:2 1800k  ** REPEAT THE ABOVE STEPS FOR THE REST OF THE OUTPUT TRACKS

    -minrate:v:2 1800k -maxrate:v:2 1800k -s:v:2 854x480

    -map 0:v -b:v:3 1000k -minrate:v:3 1000k -maxrate:v:3 1000k -s:v:3 640x480

    -map 0:v -b:v:4 600k -minrate:v:4 600k -maxrate:v:4 600k -s:v:4 480x360

    -map 0:a:0    ** FINALLY TAKE THE SOURCE AUDIO FROM THE FIRST SOURCE AUDIO TRACK.

     http://<yourchannel>.myradarmedia.channel.mediaservices.windows.net/ingest.isml/Streams(stream0^)

    • Marked as answer by Windyzje Sunday, April 1, 2018 9:21 PM
    Thursday, March 22, 2018 8:26 PM

All replies

  • Not sure exactly what you need here, but I would not recommend using the Azure Media services .NET SDK on the PI.  It is not .NET CORE.  

    If you want to use Media Services APIs from the PI, then look at using the raw REST APIs. We have a POSTMAN collection up here.

    In terms of streaming, you will need to send the content as RTMP or SMOOTH.  
    If using FFMPEG, I would recommend using Smooth format.

    Looking at my own old notes from Raspberry Pi, i think I got this to work once. You will need to play with the bitrate and scaling to optimize the CPU.  

    Note that I show in this example how to use the Hardware OMX optimizations. 

    To Enable hardware acceleration options:

    • add “–enable-omx –enable-omx-rpi” to ./configure options
    1. use encoder option “-c:v h264_omx”

    <cite style="margin:0in;margin-font-family:Calibri;font-size:9.0pt;color:#595959;">      From <https://ubuntu-mate.community/t/hardware-h264-video-encoding-with-libav-openmax-il/4997/11> </cite>

    SMOOTH STREAMING BROADCAST

    ffmpeg  -i /dev/video1 -pix_fmt yuv420p -f ismv -movflags isml+frag_keyframe  -video_track_timescale 10000000 -frag_duration 2000000 -framerate 30 -r 30  -c:v h264_omx -preset ultrafast -map 0:v:0  -b:v:0 2000k -minrate:v:0 2000k -maxrate:v:0 2000k -bufsize 2500k  -s:v:0 640x360  -map 0:v:0  -b:v:1 500k -minrate:v:1 500k -maxrate:v:1 500k -s:v:1 480x360 -g 60 -keyint_min 60 -sc_threshold 0  -c:a libfaac -ab 48k  -map 0:a? -threads 0 "http://XXXXXXXX.channel.mediaservices.windows.net/ingest.isml/Streams(video)"

     

    EXPLANATION OF WHAT IS GOING ON ABOVE ON IN A TYPICAL FFMPEG COMMAND LINE.

     

    ffmpeg

     

    -re     **READ INPUT AT NATIVE FRAMERATE

    -stream_loop -1  **LOOP INFINITE

    -i C:\Video\tears_of_steel_1080p.mov   **INPUT FILE IS THIS MOV FILE - USE the Device flags later once you test

    -movflags isml+frag_keyframe  **OUTPUT IS SMOOTH STREAMING THIS SETS THE FLAGS

    -f ismv  **OUTPUT ISMV SMOOTH

    -threads 0  ** SETS THE THREAD COUNT TO USE FOR ALL STREAMS. YOU CAN USE A STREAM SPECIFIC COUNT AS WELL

    -c:a aac  ** SET TO AAC CODEC

    -ac 2   ** SET THE OUTPUT TO STEREO

    -b:a 64k ** SET THE BITRATE FOR THE AUDIO

     -c:v libx264  ** SET THE VIDEO CODEC

    -preset fast ** USE THE FAST PRESET FOR X246, play around with other options.

     -profile:v main **USE THE MAIN PROFILE

    -g 48 ** GOP SIZE IS 48 frames

     -keyint_min 48 ** KEY INTERVAL IS SET TO 48 FRAMES

    -sc_threshold 0  ** NOT SURE WHY THIS IS SET. 

    -map 0:v   ** MAP THE FIRST VIDEO TRACK OF THE FIRST INPUT FILE

    -b:v:0 5000k   **SET THE OUTPUT TRACK 0 BITRATE

    -minrate:v:0 5000k  ** SET OUTPUT TRACK 0 MIN RATE TO SIMULATE CBR

    -maxrate:v:0 5000k  ** SET OUTPUT TRACK 0 MAX RATE TO SIMULATE CBR

    -s:v:0 1920x1080  **SCALE THE OUTPUT OF TRACK 0 to 1920x1080.

    -map 0:v  ** MAP THE FIRST VIDEO TRACK OF THE FIRST INPUT FILE

    -b:v:1 3000k ** SET THE OUTPUT TRACK 1 BITRATE TO 3Mbps

    -minrate:v:1 3000k -maxrate:v:1 3000k  ** SET THE MIN AND MAX RATE TO SIMULATE CBR OUTPU

    -s:v:1 1280x720  ** SCALE THE OUTPUT OF TRACK 1 to 1280x720

    -map 0:v -b:v:2 1800k  ** REPEAT THE ABOVE STEPS FOR THE REST OF THE OUTPUT TRACKS

    -minrate:v:2 1800k -maxrate:v:2 1800k -s:v:2 854x480

    -map 0:v -b:v:3 1000k -minrate:v:3 1000k -maxrate:v:3 1000k -s:v:3 640x480

    -map 0:v -b:v:4 600k -minrate:v:4 600k -maxrate:v:4 600k -s:v:4 480x360

    -map 0:a:0    ** FINALLY TAKE THE SOURCE AUDIO FROM THE FIRST SOURCE AUDIO TRACK.

     http://<yourchannel>.myradarmedia.channel.mediaservices.windows.net/ingest.isml/Streams(stream0^)

    • Marked as answer by Windyzje Sunday, April 1, 2018 9:21 PM
    Thursday, March 22, 2018 8:26 PM
  • Thanks john

    i already managed to upload simple static files (mainly images and videos with the help of Windows.Media.Capture) to azure with WindowsAzure.Storage package, connecting to the IoT Hub, hooked up a motion sensor (HC_SR501 with gpio) that detects motion and starts capturing media, and this is the last requirement :) but this last one seems like a big task compared to the others, since live streaming is quite complex after all, even if you "only" have to call the api in the correct orders. (i will need a mobile client too to consume the stream ofc, but thats another story, maybe you can help me start with some tips and tricks :))

    So to sum it all up: Basically i should install ffmpeg into the win 10 iot core on the raspberry pi (3), then i can call this "magic command" you described and everything should work fine? i guess the requirements for making that ingest url is that i need to open a channel, a live stream endpoint, and a program (looking at the azure portal)?

    Friday, March 23, 2018 1:04 AM
  • Where can one get FFMPEG that runs on windows iot? I'm trying to do something similar.

    Were you able to successfully get this working?
    Tuesday, November 20, 2018 3:05 PM