none
Kinect V2 and Unity SDK hangs when using DepthFrame.CopyFrameDataToArray() RRS feed

  • Question

  • Hello,

    I am using DepthSourceReader to build depth map of a surface and want to take average values over several frames for each pixel.

    Problem is that on first frame when it calls BuildTemplate and depthFrame.CopyFrameDataToArray(currentDepthData); it executes fast (about 10ms)

    On second frame it executes samely fast, but after exiting Update() it hangs for about 3 minutes, I can't debug further and just look in Task Manager - there Unity keeps 1 CPU fully loaded and KinectService load varies between 2-4% (12 cores PC).

    If I change CopyFrameDataToArray to currentDepthData = new ushort[depthReader.DepthFrameSource.FrameDescription.LengthInPixels]; than it doesn't hang

    So I use following code:

    ushort[] templateFrameData;
    ushort[] currentDepthData = new ushort[depthReader.DepthFrameSource.FrameDescription.LengthInPixels];
    int framesLeft;
    void Update () {
      if (doTemplateGeneration) {
        PrepareBuildTemplate();
      }
      if (isDoingTemplateGeneration) {
        BuildTemplate();
      }
    }
    void PrepareBuildTemplate() {
      doTemplateGeneration = false;
      isDoingTemplateGeneration = true;
      templateFrameData = new ushort[depthReader.DepthFrameSource.FrameDescription.LengthInPixels];
      framesLeft = framesForTemplate;
    }
    void BuildTemplate() {
      if (framesLeft > 0) {
        var depthFrame = depthReader.AcquireLatestFrame();
        if (depthFrame != null) {      
          depthFrame.CopyFrameDataToArray(currentDepthData);
          depthFrame.Dispose();
          depthFrame = null;
    
          for (int i = 0; i < currentDepthData.Length; i++) {
            templateFrameData[i] += currentDepthData[i];
          }
          framesLeft--;
        }
      } else {
        for (int i = 0; i < templateFrameData.Length; i++)
          templateFrameData[i] = (ushort)(templateFrameData[i] / framesForTemplate);
          isDoingTemplateGeneration = false;
        }
    }


    Tuesday, July 14, 2015 12:26 PM

Answers

All replies

  • If your code is hanging it's most likely because you need to dispose a frame somewhere.

    I'd check that your copyFrameData actually returns successfully, and if not dispose the frame.


    Sr. Enterprise Architect | Trainer | Consultant | MCT | MCSD | MCPD | SharePoint TS | MS Virtual TS |Windows 8 App Store Developer | Linux Gentoo Geek | Raspberry Pi Owner | Micro .Net Developer | Kinect For Windows Device Developer |blog: http://dgoins.wordpress.com

    Tuesday, July 14, 2015 4:37 PM
  • I digged more on this and now it seems that code is hanging if I copy or add values from copyFrameData result array to another array.

    If I just copy array/dispose frame/set frame ref to null then it doesn't hang.

    It also doesn't hang if I iterate over array and do something like:

                    for (int i = 0; i < currentDepthData.Length; i++) {
                        int tmp = currentDepthData[i] * 2;
                    }

    or even 

                    var sb = new System.Text.StringBuilder(currentDepthData.Length);
                    for (int i = 0; i < currentDepthData.Length; i++) {
                        sb.Append(currentDepthData[i].ToString());
                        sb.Append(" ");
                    }

                    Debug.Log(sb.ToString().Length);

    But it does hang if I copy/add values in some way like:

    for (int i = 0; i < currentDepthData.Length; i++) {
                        templateFrameData[i] = currentDepthData[i];
    }

    or:

    for (int i = 0; i < currentDepthData.Length; i++) {
                        templateFrameData[i] = System.Convert.ToUInt16(currentDepthData[i].ToString());
    }

    or: 

    System.Array.Copy(currentDepthData, templateFrameData, currentDepthData.Length);

    Wednesday, July 15, 2015 4:58 AM
  • Found it, array was public and therefore it was Unity inspector which hangs on serialization. 

    Fixed it by adding get/set property for it.

    Wednesday, July 15, 2015 8:22 AM