none
Writing depth data to text files RRS feed

  • Question

  • Hi 

    I am new to programming and Kinect. I need help on storing depth data to a text file at the fastest possible rate. 

    I have done multiple object serialisation to a text file. However, it takes ages to deserialise the data. 

    Please advice.

    Thanks.

    Sunday, July 17, 2011 8:31 AM

Answers

  • This is what I use, I have not benchmarked the framerate, but it seems to work okay.

    TextWriter tw = new StreamWriter("Depthframe" + (totalFrames) + ".txt");
            for (int i = 0; i < depth.Length; i++)
            {
              tw.WriteLine(depth[i]+" ");
            }
            tw.Close();
    

    It just creates a really long bit array. 

    (Streamwriter is from the class System.IO.TextWriter you would have to add "using System.IO;")

    hope I have helped. 

    Monday, July 18, 2011 3:15 AM

All replies

  • This is what I use, I have not benchmarked the framerate, but it seems to work okay.

    TextWriter tw = new StreamWriter("Depthframe" + (totalFrames) + ".txt");
            for (int i = 0; i < depth.Length; i++)
            {
              tw.WriteLine(depth[i]+" ");
            }
            tw.Close();
    

    It just creates a really long bit array. 

    (Streamwriter is from the class System.IO.TextWriter you would have to add "using System.IO;")

    hope I have helped. 

    Monday, July 18, 2011 3:15 AM
  • Thanks a lot for such a detailed reply. Actually now my boss wants to do it without using deserialisation and it is actually faster than it were using serialisation.

    But thanks anyways!

     

     


    Anu Aggarwal
    Monday, July 18, 2011 2:42 PM
  • Anu,

    So, are you still asking for support regarding the new way you need to do things, or is your issue resolved already?

    Eddy


    I'm here to help
    Monday, July 18, 2011 9:31 PM
  • Thanks, Eddy.

    It is resolved. What I am doing is now writing each image's data to a separate file and that does not take too long.

    However, I have another question - could you please advice a little on where to search for documentation regarding polling?

    Thanks in advance


    Anu Aggarwal
    Tuesday, July 19, 2011 5:06 AM
  • Anu,

    About polling, do you mean polling for frames as described in solution for this thread: http://social.msdn.microsoft.com/Forums/en-US/kinectsdk/thread/623aa311-8626-42b8-ac4f-d06ee679bd76?

    Eddy


    I'm here to help
    Wednesday, July 20, 2011 10:16 PM
  • Hi Eddy,

    Thanks for that link. I have tried something using that link. However, I am getting the following error:

    Object reference not set to an instance of an object.

    Here's the code I'm using:

     private void Window_Loaded(object sender, RoutedEventArgs e)

            {  // initialise the Nui API runtime 

                nui.Initialize(RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseColor);

               //open streams

                nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);

                nui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);

                ImageFrame iFrame = new ImageFrame();

                iFrame = nui.DepthStream.GetNextFrame(0);

                PlanarImage image = new PlanarImage();

                image = iFrame.Image;

                while(true)

                {

                    iFrame =  nui.DepthStream.GetNextFrame(1000);

                    image = iFrame.Image;

    }

    }

    Is this the way to go? Am I doing something wrong??

    Thanks in advance.


    Anu Aggarwal
    Thursday, July 21, 2011 5:28 AM
  • That error means that you're using an object on the left side of a "." that hasn't been initialized to an instance of that type of object. Run your code under a debugger and see which variable is being accessed at the line where the error break happens, and you'll know which variable you need to initialize.
    Thursday, July 21, 2011 5:38 PM
  • I thought I did understand the meaning of the error. However, what I do not understand is why I am getting it when I think I have initialised all 

    I am getting the error on the highlighted line below:

     private void Window_Loaded(object sender, RoutedEventArgs e)

            {  // initialise the Nui API runtime 

                nui.Initialize(RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseColor);

               //open streams

                nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);

                nui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex);

                ImageFrame iFrame = new ImageFrame();

                iFrame = nui.DepthStream.GetNextFrame(0);

                PlanarImage image = new PlanarImage();

                image = iFrame.Image;

                while(true)

                {

                    iFrame =  nui.DepthStream.GetNextFrame(1000);

                    image = iFrame.Image;

    }

    }

     

    Thanks in advance!


    Anu Aggarwal
    Thursday, July 21, 2011 5:46 PM
  • Anu,

    GetNextFrame method might return null if there is no new frame to return. You need to check

    if (iFrame != null)
    {
      image = iFrame.Image;
      ...
    }

    before accessing members of iFrame. Try increasing the wait time on the first call to GetNextFrame from zero to something like 200 or 1000, and it will be less likely that first call will return null, but you still have to check for null, since it is possible that no frames will have been received in the timeframe you specified.

    Hope this helps,
    Eddy


    I'm here to help
    Thursday, July 21, 2011 7:57 PM
  • Yeah, thanks a ton! It has helped resolve the problem of data collection. However, is there a way to display these images on to the image in main window? 

    As it is done for event triggered mode?

    I am not able to update the display onto the depth image holder. 

    I am using

    depth.Source = iframe.ToBitMapSource();

    And iframe is updated as in the code above.

    Thanks

     


    Anu Aggarwal
    Friday, July 22, 2011 9:37 AM
  • I would expect this to work just the same as for event triggered mode. Are you using the exact same method for displaying depth frame in one method vs the other? Also, have you tried both the ToBitMapSource() method and the procedure used by nui_DepthFrameReady method in SkeletalViewer sample (C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CS)?

    Let me know,
    Eddy


    I'm here to help
    Friday, July 22, 2011 8:13 PM
  • For depth source, when manually creating bitmap without doing your own mapping to false color, you should do the following:

     

    depth.Source = BitmapSource.Create(Image.Width, Image.Height, 96, 96, PixelFormats.Bgr565,null, Image.Bits, Image.Width * Image.BytesPerPixel);
    

    Since the depth stream contains 2 bytes per pixel. If that still doesn't work, try stepping through collectData function or outputting value of BitmapSource.Create to console before assigning it to depth.Source, to see if maybe null is being output or something like that.

     

    Eddy


    I'm here to help
    Tuesday, July 26, 2011 12:32 AM
  • HI Eddy,

     

    Thanks for the correction! I changed the last argument to the function. However, why should it be bgr565? I thought it would be gray16.

    I also stepped through the function which is fine - I mean no null output but still images are not being updated.

    Shall be grateful if you could help!

     


    Anu Aggarwal
    Tuesday, July 26, 2011 6:40 AM
  • Well, the data is in 16 bits, in fact, but not all 16 bits are used for depth data, so trying to use all 16 bits for grayscale will probably end up with an unexpectedly dark image unless you scale it pixel by pixel. Also, I didn't have luck with getting WPF to display any Gray16 images in my system. Gray8 images do work, but then you'd have to allocate a whole new array that has 1 byte per pixel and where you scale the depth values from having a 12 bit range to having a 8 bit range.

    I give Bgr565 as an example because it's the one that requires least code as an example, and it gives good enough results to see that things are working. If you want to do a nicer false color mapping then you could use Bgr32 (4 bytes per pixel) and calculate appropriate mapping from one space to another yourself, pixel by pixel, and if you want to do grayscale you can do what I suggest above, also pixel by pixel, but both require more work to just see things working on screen than just using Bgr565.

    Look at convertDepthFrame method in MainWindow.xaml.cs in C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CS sample to see an example of how to do a false color mapping from depth+player index stream.

    Eddy


    I'm here to help
    Tuesday, July 26, 2011 9:01 AM
  • Hi

     

    yeah that did occur - i mean darker image than usual. 

    It does not matter much for my project as I am doing image analysis in Matlab. But I do need to capture data a bit more reliably. So, it would be better if I can visualise what I am saving and writing to the files. So, I would appreciate if there is a way I could see the images on the image holder.

    Its ok even if it is not but if it is it will be very nice.

     

    Thanks


    Anu Aggarwal
    Tuesday, July 26, 2011 11:59 AM
  • if you're doing visualization in Matlab, you can probably multiply entire vector/matrix storing image pixels doing something like:

    scaled = 2*image

    And keep it in grayscale, just rescaled. Multiplication factor is 2 because for an image that comes from depth+player index, lowest 3 bits are used for player index, highest order bit is unused and remaining 12 bits have depth value, so multiplying by 2 or left shifting by 1 would do the trick.

    Eddy


    I'm here to help
    Tuesday, July 26, 2011 10:26 PM
  • Hi Eddy,

     

    Apologies for I forgot to mention that I am using converted distance for each pixel as the value of the image.

    So, I do not think I need to multiply or divide it. Is that correct?

     

    Thanks


    Anu Aggarwal
    Wednesday, July 27, 2011 11:05 AM
  • Oh, I didn't see any conversion code in your code above. Does that mean that for your depth image you have some code that, pixel by pixel, does something like this:

    // lower 12 bits of 'depth' variable contains depth data
    short depth = (Image.Bits[i+1] << 5) | (Image.Bits[i] >> 3);
    // Assign to a pre-allocated image array that has 1 byte per pixel, for 8-bit channel grayscale image
    targetImage[i/2] = (byte)(0xff*depth/0xfff);

    Eddy 


    I'm here to help
    Wednesday, July 27, 2011 6:22 PM
  • Apologies for not having posted that or mentioned that but yeah, it does something like that you have mentioned above. It is from one of the online demos.

    Thanks,

    AA

     

     


    Anu Aggarwal
    Wednesday, July 27, 2011 8:01 PM
  • OK. Then about your problem of why things don't display, I'm not sure. Have you stepped into debugger to see what values are returned for iFrame and vFrame every time? I guess if you didn't want to alter timing characteristics, you could do something like (temporarily, for debugging) add 3 variables to MainWindow class called:

    int numValidDepthFrames = 0;
    int numValidVideoFrames = 0;
    int numValidBothDepthAndVideoFrames = 0;


    then in your collectData method you could have, after getting various frames:

    if (iFrame != null) numValidDepthFrames++;
    if (vFrame != null) numValidVideoFrames++;
    if ((vFrame != null) && (iFrame != null)) numValidBothDepthAndVideoFrames++;
    


    then in button1_Click_1, after the collectData loop you could output

    System.Console.Write("Depth: {0}, Video: {1}, DepthAndVideo: {2}", numValidDepthFrames, numValidVideoFrames, numValidBothDepthAndVideoFrames);

    And let me know what the output is.
    Eddy


    I'm here to help
    Wednesday, July 27, 2011 9:16 PM
  • Hi, JosephKinect

     

    I am a new Kinect SDK, could you tell me how to use your code 

    TextWriter tw = new StreamWriter("Depthframe" + (totalFrames) + ".txt");
        for (int i = 0; i < depth.Length; i++)
        {
         tw.WriteLine(depth[i]+" ");
        }
        tw.Close();

    to write a depth map based on the sample code skeletalViewer ?

     

    Thank you very much !

    Song

     

     

    Wednesday, August 24, 2011 4:23 PM
  • Hi JosephKinect,

    when I use your code I have got two errors:

    Error 1 'System.Windows.Controls.Image' does not contain a definition for 'Length' and no extension method 'Length' accepting a first argument of type 'System.Windows.Controls.Image' could be found (are you missing a using directive or an assembly reference?) G:\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CS\MainWindow.xaml.cs 191 39 SkeletalViewer

    Error 2 Cannot apply indexing with [] to an expression of type 'System.Windows.Controls.Image' G:\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CS\MainWindow.xaml.cs 193 30 SkeletalViewer

    could you tell me how can I fix them? please...

    Saleh


    Best Regards
    Monday, September 12, 2011 1:36 AM