locked
MF Video Media Source Example RRS feed

  • Question

  • Are there any VideoSource examples for MF SDK such as WavSource (But for video: multiple streams)?
    Wednesday, April 4, 2007 3:16 AM

Answers

  • We don't provide an SDK sample for this, and it does involve a little elbow grease.  Here are a few concepts that a dynamic Media Source needs to deal with (a fixed-stream Media Source doesn't need to do any of this, of course).  I'll try my best to cover everything you need to know (I'm not documentation :-) )

     

    In our example, let's say that stream1 is there for the entire presentation, but stream2 appears somewhere in the middle:

    • Instead of just building an IMFPresentationDescriptor to describe what your Media Source does, it builds one IMFPresentationDescriptor for each "segment", i.e. one will contain only stream1, and one will contain only stream2.  You need not have built all of your IMFPresentationDescriptors up-front.
    • As you're playing the stream1-only part, your Media Source should send the MENewPresentation event with the Presentation Descriptor that has stream1 and stream2.  It doesn't really matter when you send this, so long as the application will have enough time to resolve the new topology and queue it before it's time for that second segment to play (otherwise, you'll get glitching)
    • As you are approaching the end of segment1, you will need to send MEEndOfStream on stream1 and MEEndOfPresentationSegment from your Media Source.  (MEEndOfPresentation should be used at the end of segment2, if that is your last segment.)
    • When it is time to transition from segment1 to segment2, the MF pipeline will make an IMFMediaSource:Tongue Tiedtart() call with VT_EMPTY.  This means that you should start segment2.  You'll need to send all the usual "started" events (with MEUpdatedStream for stream1 and MENewStream for stream2).
    • Timestamps should start at 0 for each segment.  If you're running in the MF pipeline, the MF pipeline will take care of adjusting your timestamps so that they appear to be increasing continuously across segment boundaries. 
    • Seeking is probably going to be where you need to do the most work: 
      • When you're given a seek time, you'll have to know whether it's going to be in segment1 or segment2.  If you control the application, the best way to do this is to define your own custom time format, and have this tell you which segment it's trying to get you to seek to.
      • Suppose you are playing segment1, and the caller calls Start() requesting a seek to somewhere in segment3.  You will first send all the usual events for a seek (MESourceSeeked and friends).  Then you need to end segment1 (via MEEndOfStream / MEEndOfPresentationSegment).  It's possible (likely) that you have already announced segment2 via MENewPresentation, even though you are no longer interested in playing segment2 due to the seek.  The caller is nevertheless supposed to call Start( VT_EMPTY ) upon receiving your EndOfPresentationSegment events for stream1, and you need to start segment2, announce segment3 via MENewPresentation, and then immediately end segment2.  This is known as a "fake start"; for these presentations, the MESourceStarted event should have the MF_EVENT_SOURCE_FAKE_START attribute set to 1, and the MEEndOfPresentationSegment event should have the MF_EVENT_SOURCE_TOPOLOGY_CANCELED attribute set to 1.  The caller will then call Start( VT_EMPTY ), and you should start segment3 as requested.

     

    In summary, it's not trivial, but we've done it in-house, and if you're sufficiently motivated, so can you.

    Monday, June 4, 2007 7:58 PM

All replies

  • No, we haven't published a sample video source in the SDK, or a source with multiple streams. (I agree this would useful to have.)

     

     

    ----------------------------------------------------------------------------
    Mike Wasson, SDK Documentation
    This posting is provided "AS IS" with no warranties, and confers no rights. You assume all risk for your use.

    (c) 2007 Microsoft Corporation. All rights reserved.

     

    Wednesday, April 4, 2007 5:25 PM
  • I had a couple of questions then.

     

    In a media source, do you have to know exactly how many streams/media types there are, or can you just default it to 1 (wihtout knowing media type) and later in a demux MFT either add another stream or get the media types?

     

    Wednesday, April 4, 2007 9:53 PM
  • It's quite a bit easier if your Media Source can know up front how many streams there are.  For some formats, it's possible to grovel through the beginning of a file and figure that out; this is definitely the recommended way of doing it.

     

    Putting a demux into the topology probably isn't the way to go, since we don't currently have a way of letting MFTs dynamically add streams.

     

    Having said all that, it is possible to write a Media Source that dynamically adds a stream -- if you really do need to.  The idea is that your Media Source would be one that deals with multiple presentations: Specifically, it would send MENewPresentation to announce the new Presentation Descriptor (with the new stream), pay attention to the IMFPresentationDescriptor * argument to IMFMediaSource :: Start, and MEEndOfPresentationSegment to end the current segment and move onto the new one.  It will all work, but the devil here is in the details; if you're considering going this route, reply to the post, and I'll try to give more details.  (I'll warn that my newsgroup presence is flaky these days because I had a baby a couple weeks ago and am on maternity leave right now.)  This, while certainly possible (I've done it), is certainly more work than groveling the beginning of the file and getting the streams right up-front, if you can do it.

     

     

     

    Thursday, April 5, 2007 12:47 PM
  • Hi Becky,

    Sure i'd like to know. But if you don't need to dynamically add streams, then wouldn't a source->demux be fine?

    Thursday, April 5, 2007 5:15 PM
  • Sure, but here's the thing: Your demux would have to know how many streams and what their media types are* before streaming starts, in which case you may as well have your Media Source do the demuxing (for what it's worth, our ASF Media Source does demuxing inside the source).  By "dynamically adding streams" I meant adding a stream after you've seen some data, i.e. after starting.

     

    * I suppose you could do something like declare the stream up-front from your demux with a guess at the media type and then issue a media type change from the demux MFT when you start streaming.  But of course then you'd need to know that the downstream components will be able to handle whatever media type change that involves dynamically.

     

     

    Monday, April 16, 2007 7:43 PM
  • Hi Becky,

    Thanks for the info. Can you give me more detail about dynamically adding streams in the Source via new Presentations?

    Monday, April 16, 2007 9:45 PM
  • We don't provide an SDK sample for this, and it does involve a little elbow grease.  Here are a few concepts that a dynamic Media Source needs to deal with (a fixed-stream Media Source doesn't need to do any of this, of course).  I'll try my best to cover everything you need to know (I'm not documentation :-) )

     

    In our example, let's say that stream1 is there for the entire presentation, but stream2 appears somewhere in the middle:

    • Instead of just building an IMFPresentationDescriptor to describe what your Media Source does, it builds one IMFPresentationDescriptor for each "segment", i.e. one will contain only stream1, and one will contain only stream2.  You need not have built all of your IMFPresentationDescriptors up-front.
    • As you're playing the stream1-only part, your Media Source should send the MENewPresentation event with the Presentation Descriptor that has stream1 and stream2.  It doesn't really matter when you send this, so long as the application will have enough time to resolve the new topology and queue it before it's time for that second segment to play (otherwise, you'll get glitching)
    • As you are approaching the end of segment1, you will need to send MEEndOfStream on stream1 and MEEndOfPresentationSegment from your Media Source.  (MEEndOfPresentation should be used at the end of segment2, if that is your last segment.)
    • When it is time to transition from segment1 to segment2, the MF pipeline will make an IMFMediaSource:Tongue Tiedtart() call with VT_EMPTY.  This means that you should start segment2.  You'll need to send all the usual "started" events (with MEUpdatedStream for stream1 and MENewStream for stream2).
    • Timestamps should start at 0 for each segment.  If you're running in the MF pipeline, the MF pipeline will take care of adjusting your timestamps so that they appear to be increasing continuously across segment boundaries. 
    • Seeking is probably going to be where you need to do the most work: 
      • When you're given a seek time, you'll have to know whether it's going to be in segment1 or segment2.  If you control the application, the best way to do this is to define your own custom time format, and have this tell you which segment it's trying to get you to seek to.
      • Suppose you are playing segment1, and the caller calls Start() requesting a seek to somewhere in segment3.  You will first send all the usual events for a seek (MESourceSeeked and friends).  Then you need to end segment1 (via MEEndOfStream / MEEndOfPresentationSegment).  It's possible (likely) that you have already announced segment2 via MENewPresentation, even though you are no longer interested in playing segment2 due to the seek.  The caller is nevertheless supposed to call Start( VT_EMPTY ) upon receiving your EndOfPresentationSegment events for stream1, and you need to start segment2, announce segment3 via MENewPresentation, and then immediately end segment2.  This is known as a "fake start"; for these presentations, the MESourceStarted event should have the MF_EVENT_SOURCE_FAKE_START attribute set to 1, and the MEEndOfPresentationSegment event should have the MF_EVENT_SOURCE_TOPOLOGY_CANCELED attribute set to 1.  The caller will then call Start( VT_EMPTY ), and you should start segment3 as requested.

     

    In summary, it's not trivial, but we've done it in-house, and if you're sufficiently motivated, so can you.

    Monday, June 4, 2007 7:58 PM