locked
Unable to get video and Audio to Render at same time on Media Session RRS feed

  • Question

  • I am working on Rendering Audio and Video on Media Session. Right now, I can Render Video by itself, and Audio by itself, but when I try to render both at the same time on the same topology, I get nothing. I've tried making a topology "Branch" for each based off of each of their own media Sources, and using MFCreateAggregateSource and building based off of the major type GUID Attribute you can get from the StreamDescriptor for each stream. 

    if I have multiple inputs and outputs on branches that don't interact with each other, can the Topology Handle that?

    Thursday, March 17, 2011 11:27 PM

Answers

  • Found the answer and got video and Audio playback to work.

    Basically: Media Session cannot handle multiple Sources. I think this is shuffled away in TopoEdit, but when you need to pull from multiple sources, you make an Aggregate source which will make a single media source with multiple streams. 

    Worth Noting- you can have multiple source -Nodes-, just not multiple sources. You end up making the source node with the Media Source, it's presentation Descriptor, and the streams Stream Destriptor. 

    Solution:

    Create Aggregate Media source, and make sure that you use that when making the source node. so

    // Psudeo Code. Usually you check to make sure these will pass, and it's not a horrible idea to set the variables to NULL to make it easier to check later on.

    // if this code compiles at all(and it wont) there will be memory leaks because this is also not an example of how to clean up.

    // Also: "CreateVideoSource" and "CreateAudioSource" are fictional methods that don't exist- Just calling them

    // to skip the part of actually making the sources to get to the point of this example

     

     

    // Individual Media Sources

    IMFMediaSource* pVideoSource;

    IMFMediaSource*pAudioSource;

     

    // Aggregate Source 

    IMFCollection* pSourceCollection;

    IMFMediaSource* pAggregateCollection;

    IMFPresentationDescriptor* pPD;

    IMFStreamDescriptor* pVideoSD;

    IMFStreamDescriptor* pAudioSD;

    DWORD cStreams;

    // Make individual Sources

    CreateVideoSource(&pVideoSource); 

    CreateAudioSource(&pAudioSource);

    // Create collection and add sources

     

    CHECK_HR(MFCreateCollection(&pSourceCollection));

    CHECK_HR(pMediaSourceCollection->AddElement(pVideoSource));

    CHECK_HR(pMediaSourceCollection->AddElement(pAudioSource));

    // Create Aggregate Source out of the collection of sources

    CHECK_HR(MFCreateAggregateSource(pAggregateCollection, &pSourceCollection));

     

    // Create descriptor 1 descriptor for all Streams

    m_pSourceCollection->CreatePresentationDescriptor(&pPD);

     

    //  get the count of streams so we can check them.

    pDescriptor->GetStreamDescriptorCount(&cStreams);

    for(DWORDi = 0; i < cStreams; ++i)

    {

    IMFStreamDescriptor* pSD;

    IMFMediaTypeHandler* pMediaTypeHandler;

    IMFMediaType* pMediaType;

    BOOL fSelected;

    GUID majorType;

    // get stream, make sure it is selected.

    pDescriptor->GetStreamDescriptorByIndex(i, &fSelected, &pSD);

    if(fSelected)

    {

    // get the MediaTypeHandler to get the media type so we know what the type is

    pSD->GetMediaTypeHandler(&pMediaTypeHandler);

    pMediaTypeHandler->GetCurrentMediaType(&pMediaType);

    //Get Type

    pMediaType->GetGUID(MF_MT_MAJOR_TYPE, &type);

     

    // set the stream Descriptors.

    if(type == MFMediaType_Video)

    pVideoSource = pSD;

    else if(type == MFMediaType_Audio)

    pAudioSource = pSD;

    }

    }

    // make the source node

    // Slightly Important part. Make the nodes

    // Create the Video Source node.

    HRESULT hr;

    CHECK_HR(MFCreateTopologyNode(MF_TOPOLOGY_SOURCESTREAM_NODE, &pVideoSourceNode)); // not implemented here, &pVideoSourceNode is type IMFTopologyNode*

    // Set the attributes.

    // Note, Source is the Aggregate Source

    CHECK_HR( pNode->SetUnknown(MF_TOPONODE_SOURCE, pAggregateSource));

    // The presentation is also from the Aggregate Source

    CHECK_HR(pNode->SetUnknown(MF_TOPONODE_PRESENTATION_DESCRIPTOR, pPD));

    // The VideoStream is from the Aggregate Source, but it's the Video stream. This is how the Media Session can do Audio and Video from seperate Sources

    CHECK_HR( pNode->SetUnknown(MF_TOPONODE_STREAM_DESCRIPTOR, pVideoSD));

     

    // Create the Video Source node.

    HRESULT hr;

    CHECK_HR(MFCreateTopologyNode(MF_TOPOLOGY_SOURCESTREAM_NODE, &pAudioSourceNode)); // not implemented here, &pAudioSourceNodeis type IMFTopologyNode*

    // Set the attributes.

    // Note, Source is the Aggregate Source

    CHECK_HR( pNode->SetUnknown(MF_TOPONODE_SOURCE, pAggregateSource));

    // The presentation is also from the Aggregate Source

    CHECK_HR(pNode->SetUnknown(MF_TOPONODE_PRESENTATION_DESCRIPTOR, pPD));

    // The VideoStream is from the Aggregate Source, but it's the Audiostream. This is how the Media Session can do Audio and Video from seperate Sources

    CHECK_HR( pNode->SetUnknown(MF_TOPONODE_STREAM_DESCRIPTOR, pAudioSD));

     

    // Add nodes to topology.

    pTopology->AddNode(pVideoSourceNode);

    pTopology->AddNode(pAudioSourceNode);

     

    Friday, March 18, 2011 6:10 PM