IMFMediaEngine::TransferVideoFrame - problem with alpha-channel

Question

Hello!

I'm trying to render a video by doing the following steps:

1. create a ID3DTexture2D using this format DXGI_FORMAT_B8G8R8A8_UNORM

2. putting in that texture frames (encoded by media engine) using IMFMediaEngine::TransferVideoFrame

3. render that texture in my engine (through ID3D11DeviceContext::DrawIndexed)

And here is the problem: the picture is rippling.

We've spent queit a lot of time to locate a problem and figured that the cause of the problem is that the alpha-channel contents garbage.

By using a special shader I've got a picture of alpha-channel - it looks like this:

And it's constantly changing "the pattern".

If i turn off alpha-channel by creating texture in this format DXGI_FORMAT_B8G8R8X8_UNORM - rippling disappears.

So i wounder is it just a beta-version problem and will be fixed or what?

Or may be I'm doing somthing wrong - in this case, please help me to find the bug.

Thank you

Mike Nikolaichev

Answer 1

Hello,

Would you please upload a sample codes to skydriver?
http://skydrive.live.com/

It is not necessary that you send out the whole of your project. We just need a simplest sample to reproduce the problem. You can remove any confidential information or business details from it.

Best regards,
Jesse

---

Jesse Jiang [MSFT]
MSDN Community Support | Feedback to us

Answer 2

Hello Jesse!

thank you for you reply!

Well, it's a part of a pretty big project and it would be hard to just cut off a piece and make it work. we'll think about what we can do.

But if you'll have some idea about what we can do locally to check, or test, or fix the bug - i would love to hear it.

---

Mike Nikolaichev C++ developer at Alawar Friday's Games

Answer 3

Hello Mike,

What is your video source? There are very few modern codecs that support alpha. For example if you are using h.264 or VC1 codecs you are not going to get alpha channel support. It is very likely that since you are requesting that an RGBA texture format be used we are simply seeing uninitialized data in the alpha channel since the ME is not filling in this channel since it is missing in the incoming video stream.

I hope this helps,

James

---

Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Answer 4

Hello, James!

thanks for your reply!

So we are using h.264 and it surely doesn't have alpha-channel and we clean the target texture after creating it with zeros.

If we render video to the small region of the texture - we see alpha-channel garbage just in that part of the texture.

Since the ME should not fill the alpha-channel, and we did initialize it - we expect to see that data which we've put there. But we see garbage.

So we think that this issue is a bug in Media Engine, and the correct behavior would be: if there is no alpha-channel in the source video - it should not touch it in the target texture. Can it be fixed?

---

Mike Nikolaichev | C++ developer at Alawar Friday's Games

Answer 5

Hello Mike,

Here is the answer to your question:

Q. Can it be fixed?

A. Yes, but... I am certainly willing to report this as a bug but before I do I need a few things from you. 1) Simple sample code that reproduces the issue. 2) Why is the workaround of using a format that does not contain alpha not acceptable? 3) How does this issue negatively affects your business?

So why do I need this? As I 'm sure you can understand, at this late stage in the game our teams are trying very hard to prioritize their workload and make sure that we get all of the big issues addressed first. This seems to be a real corner case and I know they are going to ask me how it directly impacts our development community. Honestly at this point I'm not seeing a real need to get this fixed when a simple documentation update would probably suffice. The more you can give me the better chance we will have to see this get addressed.

Thanks much,

James

---

Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Answer 6

Hello James,

Thank you for your time.

About things you've asked:

1) as i previously have said: " it's a part of a pretty big project and it would be hard to just cut off a piece and make it work" - and we are busy as well, and don't realy have time to do it.

2) for this particular issue we already used format without alpha - and it works fine. But in some other cases we use video with transparency in the game - and we need to have alpha there.

3) again, as i said above : "We've spent queit a lot of time to locate a problem and figured that the cause of the problem" - that was about a week! And finding other way to do transparecy will take more time as well. And as you know - time is money.

I understand that you are busy now, working hard to everything ready for the release. And, probably, you are right - and this is not that big issue, but can you at least make some notes about this in the documentation, and schedule to fix the bug as soon as it will be possible? I would appreciate that.

Thanks in advance!

---

Mike Nikolaichev | C++ developer at Alawar Friday's Games

Answer 7

It seems alpha of "video area“ will be filled with 1.0,others 0.0, then blended to the render target texture.

No problem in my testing:

Answer 8

Hello Mike,

I have reported this issue to the team and I'll see what I can do to get the documentation updated for the next release.

Thanks,

James

---

Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Answer 9

Thank you, James!

---

Mike Nikolaichev | C++ developer at Alawar Friday's Games