none
How to print Color and IR data saved in an array to a text file? RRS feed

  • Question

  • Hi,

    I am trying to save color and IR data each to an array and print it to a text file. I am using Kinect V2 ColorBasics and InfraredBasics sample files and editing them accordingly.

    For my InfraredBasics file, I declare a variable inside 'public partial class MainWindow : Window, INotifyPropertyChanged' like follows:

    private ushort[] myIrData = new ushort[434176]; //I found this value by looking at autos values after one code execution

    Then in 'private void Reader_InfraredFrameArrived(object sender, InfraredFrameArrivedEventArgs e)' method, inside  if (infraredFrame != null) i wrote the following lines. 

    ushort[] myIrData = new ushort[infraredFrame.FrameDescription.Height * infraredFrame.FrameDescription.Width];
    infraredFrame.CopyFrameDataToArray(myIrData);

    var sb = new StringBuilder("new IR values {");
                        ///*//trying to print
                        for (int i = 2; i < myIrData.Length; i++)
                        {
                            Console.WriteLine(myIrData[i]);
                            
                            var b = myIrData[i];

                            if (i < myIrData.Length - 1)
                            {
                                sb.Append(b + ",");
                            }
                            sb.Append("}");

                            }

    Outside the 'for' loop, I have the following.

      System.IO.StreamWriter file = new System.IO.StreamWriter(@"D:\\test.txt");
                        sb.AppendFormat("{}", true);
                        //file.WriteLine(sb);
                        file.Write(sb.ToString());
                        file.Flush();

    The problem is it creates the test.txt file but do not write any data to the text file. This does not throw any exception but the text file is empty. On the output tab in visual studio I get a series of values in hundreds and thousands or ten thousands.

    So I have a few issues:

    1. What i need to do is to save the pixel data information for every frame that kinect takes in order to show the live stream when it is running. Prefereably if they are stored in an array (like one array per frame) then I can compare several arrays because my applicaiton needs to compare changes in a region of interest. How should I do this?

    2. What is the meaning of the values that get printed? Are they brightness or intensities?

    3. Why isn't the strings get printed to the text file?

    I did something similar to colorbasics file too. But it shows only 0s for all values. The difference is that I wrote a byte array for colorbasics while use ushort array for irbasics.Also I use CopyRawFrameDataToArray method for color basics.

    4. Does anyone have any idea why the color file isn't printing anything? 

    5. Could somebody please give me an idea how to store these information for each frame for a short time (for a couple of minutes) ?

    thanks

    Friday, August 26, 2016 12:46 PM

Answers

  • I changed the code and got dat file bigger than that, so I am assuming I get the data now. But, i have no way of checking whether it is really dumping the correct data. Also I am doing a uint to int conversion so I need to check that this is really dumping the actual data.

    my new code is like this:

                            byte[] myIrData = new byte[217088];
                            //infraredFrame.CopyFrameDataToArray(myIrData);
                            
                            using (var fs = new FileStream("file.dat", FileMode.Append, FileAccess.Write))
                            {
                             
                                int count = Convert.ToInt32 (infraredFrameDescription.LengthInPixels);
                                fs.Write(myIrData,0,count);
                                
                             
                            }

    This block of code appears inside

    private void Reader_InfraredFrameArrived(object sender, InfraredFrameArrivedEventArgs e)

    1. Since I am not copying the data to the array is it just garbage values that get stored in the dat file?

    2. I cannot use infraredFrame.CopyFrameDataToArray(myIrData); because it requires the argument to be ushort while fs.Write() requires a byte. Without this, since I am addressing the file stream, do you think actual data is getting dumped?

    3. I tried to find a method to read back the .dat file to see what's in it, haven't been able to do so far. Any suggestions?

    thanks a lot for your help

    1) Yes.

    2) If you check the second post from above, I have already answered about how you can deal with this.

    3) Just read from a file stream with loops and such.

    In my first answer, I asked you if you needed to record stuff and then replay them in order to use them for debug purposes, meaning testing some algorithm using the same input in order to check that the output is correct. If so, remove anything to do with writing to text files and record the IR stream through Kinect Studio. Then use Kinect Studio to play the recording back while you run your application.

    • Proposed as answer by Nikolaos Patsiouras Wednesday, August 31, 2016 1:23 PM
    • Marked as answer by Laksi11 Thursday, September 1, 2016 9:48 AM
    Wednesday, August 31, 2016 1:23 PM

All replies

  • For one, where did you get 434176? The IR frame is the same as the Depth frame which is 512*424  = 217088 px.

    1) Not sure I understand. But if you want to record a clip for reuse and be able to test your comparison algorithm, you could use Kinect Studio(comes with the SDK) and record the IR stream and the Color stream alone. Then you can replay it as many times as you want. Studio can override the Kinect service and feed you input from a playback.

    2) You should take a look here.

    "The InfraredFrame Class provides a view of the scene that looks just like a black and white photograph, but is actively lit, so brightness is consistent regardless of location and room brightness.

    So yeah, basically the Infrared frame gives you brightness back, but due to the IR sensor's light instead of the room's lights.

    Btw are you closing the file stream?

    Monday, August 29, 2016 9:53 AM
  • Hi,

    Thank you for your answer. I think I may have made a mistake there when specifying the size of array. What I thought was 512*424*2 because 2 bytes per pixel. I thought that value had to be inserted. That is how 434176 comes.

    If I am recording using kinect studio as you suggest, to use that as the input to my program, do I have to disconnect it from the machine? Should I replay it while running this code?

    I am not closing the file stream. how do i do that?

    Wednesday, August 31, 2016 10:28 AM
  • What I thought was 512*424*2 because 2 bytes per pixel. I thought that value had to be inserted. That is how 434176 comes.

    You would need that if the buffer you used to copy the data over was a byte array. But since you use a ushort array it's already 2bytes per array cell and by extension pixel.

    Wednesday, August 31, 2016 11:09 AM
  • Hi,

    One other thing. I was trying to find a method to dump the frame data to a file. My code is like this:

    byte[] myIrData = new byte[217088];
      //infraredFrame.CopyFrameDataToArray(myIrData);

    I had to comment the second line because it requires a ushort type array and the fs.Write needs a byte array.

    In contrast to the original post, you have changed the buffer to a byte array. Therefore now you should make it 434176 in length.

    You could try to pin the memory and do a CopyFrameDataToIntPtr but I think that might be a bit much for now.

    Instead , a quick and dirty solution, change myIrData back to ushort with a length of 217088. And then something like:

    static void WriteShorts(ushort[] values, string path)
        {
            using (FileStream fs = new FileStream(path, FileMode.OpenOrCreate, FileAccess.Write))
            {
                using (BinaryWriter bw = new BinaryWriter(fs))
                {
                    foreach (ushort value in values)
                    {
                        bw.Write(value);
                    }
                }
            }
        }

    Not good in terms of perf but enough to at least save some frames to disk and see if you get data.

    Wednesday, August 31, 2016 11:26 AM
  • Hi,

    One other thing. I was trying to find a method to dump the frame data to a file. My code is like this:

    byte[] myIrData = new byte[217088];
      //infraredFrame.CopyFrameDataToArray(myIrData);

    using (var fs = new FileStream("file.dat", FileMode.Append, FileAccess.Write))
               {
                     fs.Write(myIrData,0,infraredFrame.FrameDescription.Width);
                    //fs.Write(infraredFrame.GetRawPixelData(), 0, infraredFrame.PixelDataLength);//this works in v1
                }

    I had to comment the second line because it requires a ushort type array and the fs.Write needs a byte array.

    Although the code executes without any errors or exceptions, it only save about 60kB data. That means it is not even saving one frame data. The commented line fs.Write has functions that works for v1 but these functions like GetRawPixelData is not available in v2. What I want to do is to dump the frame data from the time the camera opens until it is closed. How can I write the pixel data to a file in v2? If this can be done, I may not need to save the data to arrays.

    Wednesday, August 31, 2016 12:54 PM
  • I changed the code and got dat file bigger than that, so I am assuming I get the data now. But, i have no way of checking whether it is really dumping the correct data. Also I am doing a uint to int conversion so I need to check that this is really dumping the actual data.

    my new code is like this:

                            byte[] myIrData = new byte[217088];
                            //infraredFrame.CopyFrameDataToArray(myIrData);
                            
                            using (var fs = new FileStream("file.dat", FileMode.Append, FileAccess.Write))
                            {
                             
                                int count = Convert.ToInt32 (infraredFrameDescription.LengthInPixels);
                                fs.Write(myIrData,0,count);
                                
                             
                            }

    This block of code appears inside

    private void Reader_InfraredFrameArrived(object sender, InfraredFrameArrivedEventArgs e)

    1. Since I am not copying the data to the array is it just garbage values that get stored in the dat file?

    2. I cannot use infraredFrame.CopyFrameDataToArray(myIrData); because it requires the argument to be ushort while fs.Write() requires a byte. Without this, since I am addressing the file stream, do you think actual data is getting dumped?

    3. I tried to find a method to read back the .dat file to see what's in it, haven't been able to do so far. Any suggestions?

    thanks a lot for your help

    Wednesday, August 31, 2016 1:11 PM
  • I changed the code and got dat file bigger than that, so I am assuming I get the data now. But, i have no way of checking whether it is really dumping the correct data. Also I am doing a uint to int conversion so I need to check that this is really dumping the actual data.

    my new code is like this:

                            byte[] myIrData = new byte[217088];
                            //infraredFrame.CopyFrameDataToArray(myIrData);
                            
                            using (var fs = new FileStream("file.dat", FileMode.Append, FileAccess.Write))
                            {
                             
                                int count = Convert.ToInt32 (infraredFrameDescription.LengthInPixels);
                                fs.Write(myIrData,0,count);
                                
                             
                            }

    This block of code appears inside

    private void Reader_InfraredFrameArrived(object sender, InfraredFrameArrivedEventArgs e)

    1. Since I am not copying the data to the array is it just garbage values that get stored in the dat file?

    2. I cannot use infraredFrame.CopyFrameDataToArray(myIrData); because it requires the argument to be ushort while fs.Write() requires a byte. Without this, since I am addressing the file stream, do you think actual data is getting dumped?

    3. I tried to find a method to read back the .dat file to see what's in it, haven't been able to do so far. Any suggestions?

    thanks a lot for your help

    1) Yes.

    2) If you check the second post from above, I have already answered about how you can deal with this.

    3) Just read from a file stream with loops and such.

    In my first answer, I asked you if you needed to record stuff and then replay them in order to use them for debug purposes, meaning testing some algorithm using the same input in order to check that the output is correct. If so, remove anything to do with writing to text files and record the IR stream through Kinect Studio. Then use Kinect Studio to play the recording back while you run your application.

    • Proposed as answer by Nikolaos Patsiouras Wednesday, August 31, 2016 1:23 PM
    • Marked as answer by Laksi11 Thursday, September 1, 2016 9:48 AM
    Wednesday, August 31, 2016 1:23 PM