IMFSourceReader - Retrieve all video timestampe RRS feed

  • Question

  • I am adding a WMF movie reader using IMFSourceReader to an existing commercial video app (with existing AVI and DirectShow interfaces). The app requires frame-accuracy, including the ability to rapidly determine the EXACT number of frames in a source movie (always from disk, no captures, no remote URLs) and the ability to position to a specific frame by number (not by time, non-negotiable).

    Past experience with DirectShow is that customer movie files contain all kinds of timing anomalies, so that simplistic approaches such as trying to predict frame times or counts based on the frame rate are non-starters. In AVI, frame# access was the rule. In DirectShow, it was available using TIME_FORMAT_FRAME.

    So how do I do this in WMF? (Must work with any disk movie regardless of codec and any intervening processing caused by IMFSourceReader.)

    My fallback plan is to read the entire movie a frame at a time and record each timestamp. Obviously that makes for ENORMOUSLY long startup times.

    It would accelerate the process greatly if there was a flag I could supply to ReadSample such as MF_READ_SAMPLE_TIMESTAMP_ONLY --- ie so that only the timestamp is supplied, without actually decompressing the imagery.

    Thursday, October 18, 2012 7:54 PM

All replies

  • you can use the source reader without decoding any frame.(i.e. access the raw samples)

    this shouldn´t be that expensive except IO-related stuff (depending on the speed of your disk, size of source material etc)

    Friday, October 19, 2012 6:14 AM

  • Ok, working from the raw data to acquire the times does seem to be a good speedup. Whether I use raw or decompressed data, the timestamps are still problematic though. Here's my code:

        hr = MFCreateAttributes(&pAttributes, 1);

        hr = MFCreateSourceReaderFromURL(wPath, pAttributes, &pReader);

    // Leave the media type as the original data!


    stamp.SetCount(0); while (1) { pSample = 0; flags = 0; hr = pReader->ReadSample( MF_SOURCE_READER_ANY_STREAM, // Stream index. 0, // Flags. &streamIndex, // Receives the actual stream index. &flags, // Receives status flags. &llTimeStamp, // Receives the time stamp. &pSample // Receives the sample or NULL. ); if ((flags & MF_SOURCE_READERF_ENDOFSTREAM) != 0) break; if (FAILED(hr) || !pSample) { SafeRelease(pSample); return 0; } stamp.Append(llTimeStamp); pSample->Release(); };

    And here's a hex dump of the timestamps for a sample movie. A similar result has been obtained for others.

    0x05636410  0000000000000000 0000000000051762  
    0x05636420  0000000000000000 0000000000051762  
    0x05636430  00000000000a2ec5 00000000000a2ec5  
    0x05636440  00000000000f4628 00000000000f4628  
    0x05636450  0000000000145d8a 0000000000145d8a  
    0x05636460  00000000001974ed 00000000001974ed  
    0x05636470  00000000001e8c50 00000000001e8c50  
    0x05636480  000000000023a3b2 000000000023a3b2  
    0x05636490  000000000028bb15 000000000028bb15  
    0x056364A0  00000000002dd278 00000000002dd278  
    0x056364B0  000000000032e9da 000000000032e9da  
    0x056364C0  000000000038013d 000000000038013d  

    Frame 4 and 5 have the same timestamp, as do 6 and 7, etc thereafter. I am assuming that the duplication is caused by an inappropriate deinterlacing scheme --- how can I disable that? I need monotonically increasing unique timestamps to be able to seek properly. And each frame should be presented as a frame, whether it has two interleaved fields in it or not --- I have various downstream deinterlacing methods already.

    Timestamp 0 appears as both frames 0 and 2. Timestamp 51762 appears in frame 1 and 3. I don't understand what's happening with the first 2 frames (4 fields)... that's just unacceptable.

    In general, WMF is seeming to be pretty picky, it can't read files that are read by my old AVI and DirectShow file readers (used in a fallback sequence). For example, reading an MPEG that fails at CreateSourceReaderFromURL. (Nor did it want to open a .ts that the old stuff can't, and I thought WMF was supposed to.)

    And an AVI that fails because I can't set the desired MediaType to RGB32, ARGB32, or RGB24. I'm not sure what Media Type it is when I read it back, it's not on the list (64697663-0000-0010-8000-00AA00389B71), I think it's cinepak compressed data?  I've looked at the native media types that the reader returns, and this type is the only one. I'm debugging using a 32-bit executable, so there definitely should be a cinepak codec. (Right? Right!)

    Here are some more timestamps, these from a MOV. Reads fine via my quicktime reader. In WMF I get like 3x the number of frames, with many discontinuities.

    0x056209D0  0000000000000000 0000000000034155  
    0x056209E0  0000000000000000 0000000000131805  
    0x056209F0  00000000000682aa 000000000009c400  
    0x05620A00  00000000000d0555 00000000001046aa  
    0x05620A10  0000000000138800 0000000000065d57
    0x05620A20  00000000000cbaae 0000000000262f9b  
    0x05620A30  000000000016c955 00000000001a0aaa  
    0x05620A40  00000000001d4c00 0000000000208d55  
    0x05620A50  000000000023ceaa 0000000000271000  
    0x05620A60  00000000001974ed 00000000001fd244  
    0x05620A70  0000000000394731 00000000002a5155  
    0x05620A80  00000000002d92aa 000000000030d400  
    0x05620A90  0000000000341555 00000000003756aa  
    0x05620AA0  00000000003a9800 00000000002c8cf2 

    That's completely unusable. There may be some pulldown in the original source, I don't know, and really I shouldn't have to care. There are only a certain number of displayed frames in there that everything else agrees on, and WMF should give them to me without fuss. Why should everyone else but Microsoft have to re-invent a WMF version of...

    imgp = (unsigned char*)AVIStreamGetFrame(pGet, frm);

    ...I just don't know.

    Right now, WMF is looking very much not-ready-for-prime-time. Please convince me otherwise.

    Friday, October 19, 2012 6:09 PM
  • If you are using raw samples in the source format, there is no deinterlacing going on.  The samples are just the raw frames from the input file.  By looking at the media type attribute MF_MT_INTERLACE_MODE and the sample attribute flags (see http://msdn.microsoft.com/en-us/library/windows/desktop/aa367729(v=vs.85).aspx) you can tell if a sample corresponds to a single interlaced field or a full frame.  For interlaced content with single fields per sample, there will be two samples with the same timestamp.  The timestamps are just as they are in the source content -- if the content has corruptions, so will the samples that come out of the source reader in raw form.

    MF does not support a number of legacy formats.  I do not think any Microsoft media API supports the Cinepak codec out of the box; much like with DirectShow you would need a third party codec for that.  MOV files are supported through the MPEG4 source, but MOV is not specifically supported where it deviates from the MPEG4 spec.

    MF is first and foremost a media playback and transcoding API.  It currently lacks many of the power tools that help with content creation and editing.  Editing applications can be built on top of MF, but doing so will require dealing with many of the nitty-gritty details of media processing, like single field frames and timestamp anomalies.

    Tuesday, November 20, 2012 12:26 AM