locked
Variable Tracking Frame Rate RRS feed

  • Question

  • The frame rate shown by the skeleton tracker sample application varies wildly on my computer. When first started, it runs at 30fps, but then decays over time down to 0-1fps. At that point, my computer is basically frozen until I shut down the application and let it rest for awhile. I am not running any other applications on the computer.

    So, I am trying to fix this problem. I have some basic questions:

    1) Could it be a sensor temperature issue? Can the kinect getting hot cause this problem? or is it most likely the skeleton tracking libraries?

    2) If it is my computer slowing down under a lengthy load, is there a way to throttle down the frame rate of the skeleton tracker? If I could run tracking at a lower frame rate to begin with, then maybe the application won't freeze up.

     

    Wednesday, August 3, 2011 8:02 PM

Answers

  • Eric-B,

    Sorry for it seeming that the thread died. We will need to investigate your problem symptoms some more before we can give you a recommendation about how to resolve your "mysterious" computer issues.

    About controlling frame rate, there is no config you can change to modify the frame rate at which driver is running, but as LilBudyWizer mentioned, it is possible to use a polling programming model instead of event driven programming model in order to fetch allocate and fetch frames in your application at whatever frame rate you need. This is described in page 19 of programming guide (http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf), and sample code (not including initialization of error checking) for this programming model follows:

    while (1)
    {
     const NUI_IMAGE_FRAME * pImageFrame = NULL;
     HRESULT hr = NuiImageStreamGetNextFrame(pVideoStreamHandle, 200, &pImageFrame);
     if( hr == E_NUI_FRAME_NO_DATA)
     {
      // No stream frame was found after waiting for
      // requested amount. Continue and try again (or, you could
      // also choose to bail out from function here).
      continue;
     }
     else if (FAILED(hr))
     {
      break;
     }
    
     // Your code to deal with data from pImageFrame
    
     NuiImageStreamReleaseFrame(pVideoStreamHandle, pImageFrame);
    }

     

    Hope this helps,
    Eddy


    I'm here to help
    Thursday, August 11, 2011 9:14 PM

All replies

  • Beta 1 or Beta 1 Refresh?

    • If Beta 1, please try Beta 1 Refresh.
    • If Beta 1 Refresh, we'll have to work with you to understand...

    Thanks,

    Rob Relyea, @rrelyea
    Kinect for Windows team, @KinectSDKTeam

    Wednesday, August 3, 2011 9:04 PM
  • I've now tried both Beta 1 and Beta 1 Refresh and demosntrated the same slow-downs. The latest slowdown dropped the program to 0Hz, at which point I quit and restarted the application. Looking at the task manager, there were no other processes acting up (e.g. no antivirus kicking in, no hidden updater, not even a web browser). After restarting, the frame rate started at 15Hz and climbed back to 30Hz after a few minutes.

    Note that I've also tried disabling the video stream to speed things up, and that doesn't seem to prevent the slow downs either.

    Also, I checked my computer specs... it is a 3GHz, Core 2 Duo with 4GB of RAM. It should be within the specified hardware requirements.


    Thursday, August 4, 2011 3:51 PM
  • Eric-B,

    Does this happen for you with both the C++ and the C# sample SkeletalViewer applications, or just with one of them. If just one, which one?

    Skeleton tracking code is more likely to cause this kind of problem than heating of kinect device.

    Also, if you launch the resource monitor (e.g.: from "Performance" tab in Windows Task Manager) and monitor the application over time, do you see CPU or memory usage going up over time, correlated with the decrease in frame rate you're seeing?

    Eddy


    I'm here to help
    Thursday, August 4, 2011 10:06 PM
  • I am using the C++ application. Since I will ultimately be integrating directly into other C++ utilities, I haven't spent the time on C#.

    As for the process behavior over time, using the processes list in taskmanager I observed that at 25+ fps, the application utilized ~80% of the CPU and 63Mb of Memory. When the application dropped to 10-15 fps, the CPU utiilization was fluctuating 90-97%. The size of the memory used doesn't change. With the CPU utilization, however, there seems to be a significant lag in connection with frame rate. Once the frame rate climbs back up to 30fps after having dipped, it takes awhile for the CPU numbers to drop. This could just be filtering in task manager, though. I'm not sure how to tell.

    Some additional comments:

    1) Disabling video DID help, contrary to my previous post. I ran the application all afternoon yesterday, and it never had to be restarted. The frame rate still drops below 10 fps, but the application recovers without human intervention now. Before disbling video, the computer would grind to a frustrating halt.

    2) The kinect is currently mounted at about eye level, 2-m away from me. It is tracking my upper bodyfrom behind a few objects, but otherwise, I don't move much while programming and I have a wall to my back so there aren't many other people.

    3) Starting up the resource monitor utility is a major drag on the computer. When testing with video running yesterday, trying to turn it on during a slow fps froze my computer repeatedly, so that's why I stuck to the simpler taskmanager interface.

     

    Friday, August 5, 2011 2:54 PM
  • This thread seems to be dying... Rather than trying to solve whatever my mysterious computer problems are, does anybody know the answer to my second question? Is there any way to lower or otherwise control the frame rate at which the skeleton tracker process is running? Instead of having it grow to fill up the entire CPU, can I maybe set an upper bound of some sort?

    Eric

    Tuesday, August 9, 2011 2:14 PM
  • Afaik, since its based on events, and the native framerate cant be changed, that wouldnt be possible. The only method that comes to my mind would be changing it to polling mode and sending the thread to sleep for a while after it grabbed the last frames.
    Tuesday, August 9, 2011 7:54 PM
  • It seems it would be worthwhile to verify there's not a problem with the streaming. You can use performance monitor to monitor the isochronous bytes/sec off the kinect camera. While streaming depth and color off the device that should be about 22 million bytes/sec. If that drops when your frame rate drops then a defective unit is a possibility. If it stays up then the frames are getting dropped after they arrive at the computer. If you need details on how to do that just ask.

    It would help to know how to reproduce this. Software is generally deterministic, same input equals same output. Hehe, well, a whole lot of variables. A good start though is how far in? That's the main thing you've identified for reproducing it, the length of time it runs. How long is that? A few minutes or a few days? I'm more than happy to try to reproduce it, but I don't want to watch it for a few days period, much less if you start having problems within 15-30 minutes.

    Tuesday, August 9, 2011 9:59 PM
  • What do you mean by changing it to polling mode? Is there a way to control how many frames the thread grabs? I can't find any reference to an alternative mode in the documentation.

    Eric

    Wednesday, August 10, 2011 3:28 PM
  • I just tried a different Kinect (we have several), and the same problem occurs. So I don't think it is a hardware problem. In general, though, even if it was, I really need to slow down the process so that I can run something else on the computer besides just the kinect process. These slowdowns happen regularly, and it takes less than an hour of running to see it happen.
    Wednesday, August 10, 2011 3:31 PM
  • Eric-B,

    Sorry for it seeming that the thread died. We will need to investigate your problem symptoms some more before we can give you a recommendation about how to resolve your "mysterious" computer issues.

    About controlling frame rate, there is no config you can change to modify the frame rate at which driver is running, but as LilBudyWizer mentioned, it is possible to use a polling programming model instead of event driven programming model in order to fetch allocate and fetch frames in your application at whatever frame rate you need. This is described in page 19 of programming guide (http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf), and sample code (not including initialization of error checking) for this programming model follows:

    while (1)
    {
     const NUI_IMAGE_FRAME * pImageFrame = NULL;
     HRESULT hr = NuiImageStreamGetNextFrame(pVideoStreamHandle, 200, &pImageFrame);
     if( hr == E_NUI_FRAME_NO_DATA)
     {
      // No stream frame was found after waiting for
      // requested amount. Continue and try again (or, you could
      // also choose to bail out from function here).
      continue;
     }
     else if (FAILED(hr))
     {
      break;
     }
    
     // Your code to deal with data from pImageFrame
    
     NuiImageStreamReleaseFrame(pVideoStreamHandle, pImageFrame);
    }

     

    Hope this helps,
    Eddy


    I'm here to help
    Thursday, August 11, 2011 9:14 PM
  • Thanks, Eddy... I think that's what I'm looking for. I'll give it a try and see what happens.

    Regarding frame rate, however, do you have any notion of how tracking performance decays with the frame rate fed into the skeleton tracker. I assume that it DOES decay, but are there any posted stats on how rapidly performance falls off?

    Thanks again,

    Eric

    Friday, August 12, 2011 1:27 PM
  • Sorry, there are no posted stats on this matter. My expectation is that skeleton detection will be just as good no matter how low frame rate gets, but that potentially a skeleton in a later frame will not be detected as being the same skeleton as in a previous frame, so detected skeleton might end up with a different tracking ID and player index, in a different skeleton slot from the 6 slots available.

    Eddy


    I'm here to help
    Friday, August 12, 2011 10:32 PM