Getting frames from camera and capturing screen for windows 10

Question

Hi ,

I am creating a UWP app for Windows 10 for which I need two things. 1) To get the camera video frames and process it using native code (using ffmpeg library). 2) To capture the screen continuously (just like a video)

The first part I achieved by using the mediacapture.GetPreviewFrameAsync() in a threadpool timer. But its not really elegant. Is there a way I could hook my frame processing function to an event which will trigger when there is a new frame available? I would prefer not to write a whole WinRT com library for that.

For the second part of screen capturing I am nowhere and couldn't find any helpful function in the SDK to achieve the result I need. Again I'm trying to capture the screen like a video stream rather than a one time screenshot capture.

I hope any solution would to be in accordance with UWP store certification policies too.

Any help or direction is appreciated.

Thank you

Answer 1

To capture the video content, you need to write a Media Foundation Transform(bacially C++), I don't think there is any existing C# API that can do such things.

Answer 2

So what about the screen capture? Which API supports that? Is it ok to go with something like ffmpeg/opencv etc for the first part?

Answer 3

You should look at the following documentation page: Record the screen with ScreenCapture

Additionally, to process the recorded frames, you may find the Win2D documentation to come in handy.

Answer 4

You should look at the following documentation page: Record the screen with ScreenCapture

Additionally, to process the recorded frames, you may find the Win2D documentation to come in handy.

However, ScreenCapture.GetForCurrentView() in "Record the screen with ScreenCapture" always fail with: 'ScreenCapture.GetForCurrentView()' threw an exception of type 'System.ArgumentException'
    Data: {System.Collections.ListDictionaryInternal}
    HResult: -2147024809
    HelpLink: null
    InnerException: null
    Message: "Value does not fall within the expected range."
    ParamName: null
    Source: "Windows"
    StackTrace: "   at Windows.Media.Capture.ScreenCapture.GetForCurrentView()". The call works on WP8.1 but not on Windows Mobile 10. Please help.

Answer 5

I'll give it a try and get back to you. Thanks for letting me know.

Answer 6

Okay, so I reached out to the owners of the screen recording API, and they clarified the situation for me.

This API was only supported on Windows Phone 8.1, and is not expected to work in Windows 10. The fact that the API was featured in the Windows 10 documentation page was likely an error on our side. If so, expect the article to be removed or annotated to indicate platform support in our next refresh of documentation content.

Again, thanks so much for bringing this to my attention, and apologies for the trouble!

Answer 7

Thanks, I don't expect it to work on desktops. But isn't it supposed to be compatible with Windows Phone 8.1 and supported on Windows 10 Mobile as documented in

https://msdn.microsoft.com/en-us/windows.media.capture.screencapture.getforcurrentview

》Requirements (Windows 10 device family)
》Device family - Mobile

》Requirements (Windows 8.x and Windows Phone 8.x)
》Minimum supported phone - Windows Phone 8.1 [Windows Phone Silverlight 8.1 and Windows Runtime apps]

Answer 8

Sorry, I should've been more clear in my last post. I meant to say that the ScreenCapture API was not expected to work on phones running Windows 10.

We made a mistake when listing Windows 10 Mobile as a platform that supports this API. Because of the way our documentation is written and published, there may be multiple pages claiming that this is the case. I'm working with the team that owns this feature (since it's outside of the scope of the Camera Team) and with our documentation writers to correct this situation as much as we can, as soon as possible.

If you have any other questions, please let me know and I'll do my best to answer them.