none
Imagestream - someone please explain RRS feed

  • Question

  • Hi, I've been trying to do a background subtraction(real time). Obviously I would need to have two frames ready to do so,

    is there a way to get out a previous frame or something? I have neither the luck nor the clue on what to do with Imagestream.

    I would appreciate it, if some one can explain this Imagestream, or any other method in depth. Thanks in advance and no puns intended. 

    Monday, July 11, 2011 4:07 AM

Answers

  • Sure:

    Runtime nui = new Runtime();
    nui.Initialize(RuntimeOptions.UseDepth);
    ImageFrame frame = nui.DepthStream.GetNextFrame(500);
    


    Does that help?
    Eddy


    I'm here to help
    Tuesday, July 12, 2011 10:22 PM

All replies

  • Check out the sample code in samples under Start > All Programs > Microsoft Kinect For Windows SDK Beta > Samples

    For two frames you need to store the previous depth stream.

    This is just the basic idea to get you started.

     

    PSEUDO CODE

    int currentDepthStream = new byte[320 * 240];

    int prieviousDepthStream = new byte[320 * 240 ];

     

    void EventDepthStreamReady(object sender, ImageFrameReadyEventArgs e)

    previousDepthStream = currentDepthStream

    currentDepthStream = getDepthImage(e.ImageFrame.Image);

    compare(previousDepthStream, currentDepthStream);

     

    void getDepthImage(Image img)

    process(img.bits);

     

    Everytime Kinect has a complete depth Image it fires an event DepthFrameReady

     

     

    nui.DepthFrameReady += new EventHandler<ImageFrameReadyEventArgs>(nui_DepthFrameReady);
    

    The event object passes (among other things)

     

    e.ImageFrame.Image.Bits

    Which contains a byte array containing player and depth data

    From the documentation

    The format of the depth data depends on whether the application specifies depth only or depth and player index at NUI initialization, as follows:

    • For depth only, the low-order 12 bits (bits 0‒11) of each pixel contain depth data, and the remaining 4 bits are unused.
    • For depth and player index, the low-order 3 bits (bits 0‒2) of each pixel contain the player index and the remaining bits contain depth data.

    because RuntimeOptions.UseDepthAndPlayerIndex was used to initialize the depth stream in nui.Initialize then e.ImageFrame.Image.Bits

    contains 2 bytes of information relating to each pixel in the depth array set out as

    bytes 0 and 1 = (01001)[010] (10101100)

    bytes 2 and 3 = (01011)[010] (10111010)

    bytes x and y = (depth)[player] (depth)

    The bytes are ordered with the Least Significant Bits being the player data.

    http://en.wikipedia.org/wiki/Least_significant_bit

    In the Sample code Masking the player data.

    int player = depthFrame16[i16] & 0x07;
    

    Code to capture the Depth Data.

    int firstByte = depthFrame16[i16 + 1] << 5 | depthFrame16[i16] >> 3;
    int secondByte = depthFrame16[i16 + 1] >> 3 * 256;
    int total = first + second;
    

    Then store these totals in your depth array

    currentDepthStream[i16] = total;


    Monday, July 11, 2011 7:12 AM
  • The skeleton tracking will provide segmentation for humans. That tells you which pixels in the depth image is part of a human and you can map that to the color image. If you want to seperate out something else you have to set up your own segmentation. One of the easiest would be seperating static and dynamic objects. There you just keep track of the greatest depth reading for each pixel. Objects that don't move will be the greatest depth for that pixel, or close to it. Depth can vary over time. The main challenge is filtering out edges of objects. Those can vary a good deal though the object it's an edge of isn't moving.

    Monday, July 11, 2011 3:45 PM
  • Joseph,

    The answers given by others in this thread about how to track multiple image frames should help you out. About the "real time" portion of your request, I can tell you that people with real time scenarios have found it easier to use ImageStream.GetNextFrame method (or NuiImageStreamGetNextFrame in C++) called in a loop or a timer, instead of getting called back when a "frame ready" event fires.

    Hope this helps,
    Eddy


    I'm here to help
    Monday, July 11, 2011 10:43 PM
  • Can you give me an example of how to use Imagestream, I am confused as how Runtime can work with imagestream.

    would this work?

    imagestream = nui.depthframe;

    Tuesday, July 12, 2011 7:11 PM
  • Sure:

    Runtime nui = new Runtime();
    nui.Initialize(RuntimeOptions.UseDepth);
    ImageFrame frame = nui.DepthStream.GetNextFrame(500);
    


    Does that help?
    Eddy


    I'm here to help
    Tuesday, July 12, 2011 10:22 PM
  • Yes, it does. thanks. 
    Wednesday, July 13, 2011 10:22 PM