Extending the measuring distance range of Kinect2? RRS feed

  • Question

  • Dear Kinect Team,

    my name is Axel Mertes and I am from Germany. I studied computer science and work as CTO for Magna Mana Production, a film & post production company I founded in 1996 with two partners.

    I am often involved in "special interest" projects outside normal film making, but for events, museums etc. including interactive installations.

    Right now I am working on an art project where we need ToF camera capabilities. I digged and dived deep into ToF cameras now and the Kinect2 seems to be a killer device compared to many industrial products. I only found one cameras/sensors that claim a better resolutions of 640x480y:

    Basler ToF camera (Panasonic sensor, formerly used in now retired D-Imager product line):

    However, this camera is very expensive and the support is minimal, nowhere near Kinect2.

    I have a few questions and even proposals and after all my research I think you might be the best contact to answer these or know at least who can do that. 

    You can send me an email to axel at magnamana point com if you like. 

    So here we go:

    1. Is the ToF camera inside the Kinect2 using the pulsed 2-phase distance measurement or continuous-wave method as explained e.g. here in the Texas Instrument whitepaper for ToF cameras? See page 2 of the following document:

    This is interesting for me to see how much external IR (sun-) light might affect operation of the Kinect2.

    2. Is there a way to extend the depth range of the Kinect2 system?

    I found rumors that in hackatons a modified firmware has been shown, that decreased the depth range, but increased the precision on a now shorter total distance. As to my understanding of ToF that requires to shorten the exposure/IR-light integration phase, thereby potentially increasing heat etc. in the camera.

    My thought is that if we extend the integration/IR light phase, we can look deeper into the room. However, as light - including IR light - decreases quadratic over distance, we might need more/brigher IR light to cover the scene, or live with a more noisy result.

    I found exactly no ToF camera that allows me to change the depth by changing the pulse wave. Maybe its a design issue, but I would wonder of that isn't bound to a clock signal being controlled somewhere in a firmware. I that would be opened up, at least within some reasonable level, it would potentially open up completely new business options, no doubts.

    Unfortunately we need to cover like 16m in range and the Kinect can only do 8m as I found.

    Is there any way to get a modified firmware or the like to change this?

    Looking forward to hearing from you.

    Mit freundlichen Grüßen,
    Best regards,

    Axel Mertes
    Friday, August 19, 2016 10:27 PM

All replies

  • I doubt anyone will see this. Kinect Support Engineers have been absent from this forum for some time now. Also no plans for any update to Kinect have been made public, at least not as far as I know.

    Monday, August 22, 2016 10:05 AM
  • Heya,

    You can swap the lens on the Kinect v2 for a telephoto lens and get better x-y resolution at a greater distance. Requires some hardware modifications. I've had some experience with the Zed Camera that would cover the distance you are looking for out of the box.

    Tuesday, August 30, 2016 8:53 PM
  • Hi Phil,

    thanks for your comment.

    I wonder how changing the lense would actually change the distance I can measure with the ToF part of the Kinect2. As far as I understand ToF (and I think I really understand ToF) changing the lense would just change the viewing angle, but not the distance.

    I have a ZED here as well, but it will not work in my scenario, beside the fact its usually too slow or needs a high-end GPU accelerated workstation to get somewhere near realtime. However, the ToF cameras lack resolution (as of now).

    What was your performance with the ZED camera?
    I need realtime results, short delays as short as possible, its for an interactive art installation.

    Maybe using IR cameras and process them similar to what the ZED does would help me.

    I hoped that someone working on the hardware/firmware would provide a trick to decrease the pulse rate of the ToF lighting/camera and increase its exposure pulse time/iteration time and thereby extending viewing range with the tradeoff of precision due to extending bit depth to a deeper range AND working with less light, as light decreases squared over distance, reflected light even more so.

    Wednesday, September 7, 2016 12:34 PM
  • I'm guessing Phil doesn't mean he can extend the max distance from 8m to something more, but just improving the resolution and fidelity of the data near the max distance of the kinect.

    I seriously doubt you can extend the distance to > 8 meters. Even if you improve the hardware, there's still a software part and that is probably hard-coded and closed off to you. So if you change a factor like max distance on the hardware and the SDK does not provide a way to set the new max distance, then it's probably all for nothing.Perhaps what Phil says is doable because you are just improving the lens and so the data received into the service is much better, but you're not changing a factor in the SDK.

    UPDATE: The above comment applies only to MS SDK. If using libfreenect or something like that, it's possible/doable.
    Wednesday, September 7, 2016 12:42 PM