locked
Efficient video manipulation without DirectX filters RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    I am working on creating an app that will apply filters to video.  I have the (webcam) video pumped into 9 canvases displayed on the screen simultaneously and was planning on just applying filters to them using CSS (e.g. blur, brightness, invert, hue-rotate etc), not realizing that (a) these are special, directX filters and that (b) directX filters are not supported in Metro.

    Is there a good way to work around this and do these manipulations efficiently?  (Efficiently enough that I can get away with using 9 of them at once on different canvases?)

    Thank you for any help you can give with this!

    Thursday, May 24, 2012 2:22 AM

Answers

  • Question
    Sign in to vote
    0
    Sign in to vote

    Try removing XMLNS in the SVG tag (do give it a height and width though).

    And, wrap the <filter> with <defs>

    Let's see if that works.

    Thursday, May 24, 2012 6:11 PM

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote

    Would I be well served to switch to C++? (yuck!)

    I should add that I am not really trying to do any 3D effects!  (Well, fisheye too, but it seems like it should be doable...)  Mostly just 2D manipulations for me though!

    Thursday, May 24, 2012 2:35 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi nosirrahcd,

    The directX filters have been replaced by some standards (opacity only) and the W3C working groups are beginning to standardize a lot more for CSS.

    However, interestingly enough, since you are applying the video to canvas, there is an interesting method by which to apply these filters using SVG.

    I will write some pseudo-code and if you need more specific code let me know.

    Within your HTML document, define the filter you want to use using the <svg> filter syntax.  Generating the correct filter can be done quickly using a free tool called InkScape.

    The low level filters than can be composited are in the W3C SVG Filters Chapter.

    <html>

    <svg>

    <defs>

    <filter id="Gaussian_Blur">
    < feGaussianBlur in="SourceGraphic" stdDeviation="5"/>
    < /filter>
    < /defs>

    <image id="filteredImage" style="filter:url(#Gaussian_Blur)"/>

    </svg>

    <video/>

    <canvas/>

    because SVG filters do not yet apply to HTML in standards you will have to do a few copies (which is costly)

    pull a frame from the video (I think you have to copy to canvas to do this)

    ctxSource1.drawImage(video1, 0, 0, videoWidth, videoHeight);

    Once you have the image data in the canvas you can then pull it then you can push it to an SVG image (here is an article for moving data to and from SVG and Canvas)

    Then push the image into the SVG container and it will apply the filter.  It's the SVG you will show (not the canvas)

    Pretty rough, and probably pretty slow.

    Let me know if this is not clear, I do have some sample code somewhere on this.

    Patrick Dengler - Microsoft

    Thursday, May 24, 2012 5:05 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Are you certain that these will work in metro?  I try to use the following code (in the future, I will replace "circle" with my canvas"):

     <svg xmlns="http://www.w3.org/2000/svg">
            <filter id="myHueRotate">
                        <feColorMatrix type="hueRotate" values="180"/>
            </filter>
            <circle filter="url(#hueRotate)" fill="red" r="30"/>
</svg>

    When I do this I get the message: "Content is not allowed between the opening and closing tags for the element 'filter'" on the <filter> line.

    I also get: "Validation (HTML5): Element 'fecolormatrix' is not supported." on the <feColorMatrix> line.

    It is probably also very likely that I am just doing something wrong!  Thank you for all of your help!

    ...Oops didn't see your edit when I posted.  I will try that.



    • Edited by nosirrahcd Thursday, May 24, 2012 6:12 PM
    Thursday, May 24, 2012 5:49 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Try removing XMLNS in the SVG tag (do give it a height and width though).

    And, wrap the <filter> with <defs>

    Let's see if that works.

    Thursday, May 24, 2012 6:11 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Any thoughts on putting a web frame or something similar into the app and letting everything render via IE?  I could just use getusermedia to pull in the webcam video...  Would that work?  This could technically all be done in a webpage, I just want it to be a metro app.
    Friday, May 25, 2012 2:11 AM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Here is the code that works:

    <html>
<head>

</head>
<body>
<svg id="svgroot"  viewbox="0 0 800 800" width="800" height="800"  preserveAspectRatio="xMidYMin">
    <defs>
<filter id="myHueRotate">
<feColorMatrix type="hueRotate" values="270"/>
</filter>
</defs>
<image id="a" filter="url(#myHueRotate)"  x="0" y="0" width="300" height="300" />
<image id="b" filter="url(#myHueRotate)"  x="300" y="0" width="300" height="300" />
<image id="c" filter="url(#myHueRotate)"  x="0" y="300" width="300" height="300" />
<image id="d" filter="url(#myHueRotate)"  x="300" y="300" width="300" height="300" />
</svg>
<canvas id="canvas" height="300" width="300"></canvas>
<video id="vid" src="movie.m4v" height="300" width="300" style="display: none" autoplay/>


<script>
  var ctx = document.getElementById('canvas').getContext('2d');
  var img = new Image();
  img.src = 'img.jpg';
  img.onload = function(){
    //ctx.drawImage(img,0,0);
    //var canvasImage = canvas.toDataURL("image/png");
    //var svgImage = document.getElementById('a');
    //svgImage.setAttributeNS("http://www.w3.org/1999/xlink", "xlink:href", canvasImage);
    draw();
  }
img.load();

function draw(){
    var ctx = document.getElementById('canvas').getContext('2d');
    var vid = document.getElementById('vid')
    ctx.drawImage(vid,0,0,300,300);
    var canvasImage = canvas.toDataURL("image/png");
    document.getElementById('a').setAttributeNS("http://www.w3.org/1999/xlink", "xlink:href", canvasImage);
    document.getElementById('b').setAttributeNS("http://www.w3.org/1999/xlink", "xlink:href", canvasImage);
    document.getElementById('c').setAttributeNS("http://www.w3.org/1999/xlink", "xlink:href", canvasImage);
    document.getElementById('d').setAttributeNS("http://www.w3.org/1999/xlink", "xlink:href", canvasImage);
    setTimeout(draw,40);
}
</script>
</body>
</html>

    Is that what you were suggesting? (If so it is slow, but potentially usable with 4 videos at a time)...

    Thanks!

    Wednesday, May 30, 2012 1:09 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    My apologies for not responding sooner.  The model you are showing is correct.  And yes, there are performance issues.  I did this same model for the Mix Demo

    http://www.youtube.com/watch?v=TQUTsxL41Nk  (4:23 into the video)

    Here I had two videos playing, copied to canvas (not copied back to SVG), but pushed onto 10 canvas.  It also have jscript animated SVG.

    Instead of setTimeOut use requestAnimationFrame and see if that speeds you up.  Jason did a great demonstration of performance tuning as well.

    http://channel9.msdn.com/events/MIX/MIX11/HTM01

    Let me know if this helps.

    Tuesday, June 5, 2012 6:33 PM