none
CoordinateMapper, deprecated KinectSensor.Map* methods and offline mode RRS feed

  • Question

  • Hello!

    I had been looking up issues for my Kinect (SDK 1.8) program and found the following thread titled "Using the CoordinateMapper offline"

    jointPoint = sensor.MapSkeletonPointToColor(joint.Position, imageFormat);
    
    //jointPoint = sensor.CoordinateMapper.MapSkeletonPointToColorPoint(joint.Position, imageFormat);
    
    //jointPoint = mapper.MapSkeletonPointToColorPoint(joint.Position, imageFormat);

    The first line uses one of the deprecated KinectSensor.Map* methods. The second gets the CoordinateMapper directly from the KinectSensor. The third one gets the mapper from the following method (similar to the other thread above)

    public CoordinateMapper getMapper()
            {
                bool kinectStarted = sensor.IsRunning;
                sensor.Start();
                while (!sensor.IsRunning) ;
    
                var tempParameters = sensor.CoordinateMapper.ColorToDepthRelationalParameters;
                byte[] parameters = new byte[tempParameters.Count];
                tempParameters.CopyTo(parameters, 0);
    
                CoordinateMapper mapper = new CoordinateMapper(parameters);
                if(!kinectStarted)
                {
                    sensor.Stop();
                }
                return mapper;
            }

    My problem is, I can't get the mapping to work offline, as in, all 3 lines of code will throw an exception if KinectSensor.IsRunning == false. This would be acceptable in the first two cases, but in the third case, I don't even use the ColorToDepthRelationalParameters provided by the KinectSensor, instead, I copy the data, and use that, to construct the mapper. However, the third line still raises the following exception with no InnerException:

    "An exception of type 'System.InvalidCastException' occurred in Microsoft.Kinect.dll but was not handled in user code

    Additional information: Nem lehet konvertálni a(z) „System.__ComObject” típusú COM-objektumot a(z) „Microsoft.Kinect.Interop.INuiCoordinateMapper” típusú illesztőfelületre. A művelet nem sikerült, mert a(z) „{618E3670-BA84-4405-898A-3FF64446157C}” azonosítójú illesztőfelület COM-összetevőjének QueryInterface hívása nem sikerült a következő hiba miatt: Ez az objektumfelület nem támogatott (A kivétel HRESULT-értéke: 0x80004002 (E_NOINTERFACE))."

    This is while on checking the ColorToDepthRelationalParameters I get

    ColorToDepthRelationalParameters = The function evaluation requires all threads to run.

    After Evaluating:

    ColorToDepthRelationalParameters = 'mapper.ColorToDepthRelationalParameters' threw an exception of type 'System.InvalidCastException'

    An even stranger error is, when I try to run my code while the KinectSensor.IsRunning == true, the first, deprecated method works, but the second and the third does not (throwing the same Exception).

    NOTE: I do not call the mapping methods in response to a KinectSensor.AllFramesReady event, the joint that I use is from a deserialized skeleton waiting to be processed and drawn. If I use the mapping methods in response to a KinectSensor frameready event, it works as intended.

    Is there a way to circumvent the (non-deprecated) mapper throwing an exception when I try to map a perfectly good Joint?

    Thanks for your Help


    • Edited by Ábel Fekete Monday, November 4, 2013 6:12 AM typo
    Monday, November 4, 2013 6:11 AM

Answers

  • In all cases, when you are using the CoordinateMapper functionality, you are using the RelationalParameters (Skeleton is derived from depth). I was not able to reproduce the same issue you had:

    http://social.msdn.microsoft.com/Forums/en-US/b33b5f4f-b2af-447f-9216-0b2829ff998a/using-the-coordinatemapper-offline?forum=kinectsdknuiapi

    An instance of the CoordinateMapper can be created from either a sensor or the data retrieved from the ColorToDepthRelationalParameters from one. In either case, the sensor itself does not need to be in a "started", but to get the data it must be "Connected". You did not show your code how you got the instance of the KinectSensor.

    As stated in the previous post, once you save the data to a file, you just need to load that back into a byte[] and create the new CoordinateMapper object.

    byte[] parameters = LoadParamsFromFile("KinectSensorOne.bin"); // some function you create to load the byte data from a file/memory
    CoordinateMapper mapper = new CoordinateMapper(parameters);
    
    SkeletonPoint point = new SkeletonPoint();
    point.X = 0.0f;
    point.Y = 0.0f;
    point.Z = 1000.0f;
    
    ColorImagePoint imagePoint = myMapper.MapSkeletonPointToColorPoint(point, ColorImageFormat.RgbResolution640x480Fps30);
    Console.WriteLine( "SkeletonPoint: ( {0},{1},{2} ) -> ColorImagePoint:( {3},{4} )"
        , point.X, point.Y, point.Z
        , imagePoint.X, imagePoint.Y);
    

    Carmine Sirignano - MSFT


    Monday, November 4, 2013 7:57 PM

All replies

  • In all cases, when you are using the CoordinateMapper functionality, you are using the RelationalParameters (Skeleton is derived from depth). I was not able to reproduce the same issue you had:

    http://social.msdn.microsoft.com/Forums/en-US/b33b5f4f-b2af-447f-9216-0b2829ff998a/using-the-coordinatemapper-offline?forum=kinectsdknuiapi

    An instance of the CoordinateMapper can be created from either a sensor or the data retrieved from the ColorToDepthRelationalParameters from one. In either case, the sensor itself does not need to be in a "started", but to get the data it must be "Connected". You did not show your code how you got the instance of the KinectSensor.

    As stated in the previous post, once you save the data to a file, you just need to load that back into a byte[] and create the new CoordinateMapper object.

    byte[] parameters = LoadParamsFromFile("KinectSensorOne.bin"); // some function you create to load the byte data from a file/memory
    CoordinateMapper mapper = new CoordinateMapper(parameters);
    
    SkeletonPoint point = new SkeletonPoint();
    point.X = 0.0f;
    point.Y = 0.0f;
    point.Z = 1000.0f;
    
    ColorImagePoint imagePoint = myMapper.MapSkeletonPointToColorPoint(point, ColorImageFormat.RgbResolution640x480Fps30);
    Console.WriteLine( "SkeletonPoint: ( {0},{1},{2} ) -> ColorImagePoint:( {3},{4} )"
        , point.X, point.Y, point.Z
        , imagePoint.X, imagePoint.Y);
    

    Carmine Sirignano - MSFT


    Monday, November 4, 2013 7:57 PM
  • Hi,

    I faced the same issue, but I figured out what the problem was: I was using the Sensor.CoordinateMapper.MapDepthPointToSkeletonPoint and Sensor.CoordinateMapper.MapSkeletonPointToColorPoint at the same time from different threads. It seems the "concurent" accessing of the coordinate mapper caused the issue.

    But it is a good question: why.

    Wednesday, December 11, 2013 7:37 PM