none
Accessing coordinate data with minimal programming experience RRS feed

  • Question

  • Hi,

    Sorry to ask this as I know there are similar questions answered already, but I'm lost with this.

    My project is basically as follows:

    I've to hang an object like a glove, in front of a plain colored wall, and using the Kinect determine the X Y Z coordinates of each of the fingertips. I can ignore/pin back the thumb to begin with.

    Can someone explain to me in simple terms if possible the best way to go about this?

    I've had a look at the quickstarts but I feel like I'm not going to get anywhere because my programming background is almost nonexistant!

    Thanks and apologies for such a novice question.

    Wednesday, June 6, 2012 4:27 PM

All replies

  • kinect is not a good place to start if you have no programming background and finger tracking doesn't exist in current sdk but I think it might be on the agenda to implement in 2.0 or later sdks.

    So you would have to use the depth stream to capture each of the fingers points which is not an easy task and might require some math skills.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://jefferycarlsonblog.blogspot.com/

    Thursday, June 7, 2012 1:38 PM
  • kinect is not a good place to start if you have no programming background and finger tracking doesn't exist in current sdk but I think it might be on the agenda to implement in 2.0 or later sdks.

    So you would have to use the depth stream to capture each of the fingers points which is not an easy task and might require some math skills.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://jefferycarlsonblog.blogspot.com/

    As a start I planned on using a method similar to that used in the "Working with depth data" quickstart video, i.e. when the depth image is displayed in red/green/blue, the glove would be displayed green and the plain wall red.

    I know it's possible to access the XYZ coordinates of each pixel but I'm not 100% how.

    Would it be possible to implement a code such as

    If (Pixel is green)

    {

    Place in one array

    }

    Else

    {

    Place in another

    }

    Then I could at least have the coordinates of the glove in one array to work with?

    I know this wouldn't work in any situation other than the glove in front of a plain wall but it is just to get me started.

    Math skills are not a problem I'm just not proficient in any particular language but I'm studying C# as much as possible and working with Visual C# 2010 express and gradually getting the hang of it.


    • Edited by Luke24dub Thursday, June 7, 2012 7:30 PM
    Thursday, June 7, 2012 7:29 PM
  • yes i dont think using x,y,z with fingers has been implemented in kinect sdk unless the team confirms it will be in next release. You have to do math for coordinates too. If someones implemented finger tracking with kinect sdk I would see what they did.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://jefferycarlsonblog.blogspot.com/


    Thursday, June 7, 2012 7:48 PM
  • I've created a 3d point cloud from http://www.i-programmer.info/ebooks/practical-windows-kinect-in-c/4126-kinect-sdk1-a-3d-point-cloud.html

    Is there a way for example to click on a point in the point cloud and display the depth of the clicked point using mousedown or similar? 

    Tuesday, June 12, 2012 2:02 PM
  • some people have taken the depth pixel data from kinect sdk and created their own finger tracking algorithm which is how they track fingers. like pixels 3 -15 repesent the specific finger im tracking.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://jefferycarlsonblog.blogspot.com/


    Tuesday, June 12, 2012 2:31 PM
  • I'm not trying to track fingers just yet. At the moment I have a point cloud, and have it set up that when I click a certain point in the canvas image, the x and y pixel coordinates are displayed in text boxes. I want to display the depth for the clicked pixel in a third text box. How would I go about this? The code I'm using is included.


    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Windows;
    using System.Windows.Controls;
    using System.Windows.Data;
    using System.Windows.Documents;
    using System.Windows.Input;
    using System.Windows.Media;
    using System.Windows.Media.Imaging;
    using System.Windows.Navigation;
    using System.Windows.Shapes;
    
    using System.Windows.Media.Media3D;
    using Microsoft.Kinect;
    
    
    namespace PointCloudWPF
    {
    
        /// <summary>
        /// Interaction logic for MainWindow.xaml
        /// </summary>
        public partial class MainWindow : Window
        {
            GeometryModel3D[] points = new GeometryModel3D[320 * 240];
            int s = 4;
            KinectSensor sensor;
    
            public MainWindow()
            {
                InitializeComponent();
            }
    
    
            private void button1_Click(object sender, RoutedEventArgs e)
            {
                DirectionalLight DirLight1 = new DirectionalLight();
                DirLight1.Color = Colors.White;
                DirLight1.Direction = new Vector3D(1, 1, 1);
    
    
                PerspectiveCamera Camera1 = new PerspectiveCamera();
                Camera1.FarPlaneDistance = 8000;
                Camera1.NearPlaneDistance = 100;
                Camera1.FieldOfView = 10;
                Camera1.Position = new Point3D(160, 120, -1000);
                Camera1.LookDirection = new Vector3D(0, 0, 1);
                Camera1.UpDirection = new Vector3D(0, -1, 0);
    
                Model3DGroup modelGroup = new Model3DGroup();
    
                int i = 0;
                for (int y = 0; y < 240; y += s)
                {
                    for (int x = 0; x < 320; x += s)
                    {
                        points[i] = Triangle(x, y, s);
                        // points[i]=MCube(x,y);
                        points[i].Transform = new TranslateTransform3D(0, 0, 0);
                        modelGroup.Children.Add(points[i]);
                        i++;
                    }
                }
    
                modelGroup.Children.Add(DirLight1);
                ModelVisual3D modelsVisual = new ModelVisual3D();
                modelsVisual.Content = modelGroup;
                Viewport3D myViewport = new Viewport3D();
                myViewport.IsHitTestVisible = false;
                myViewport.Camera = Camera1;
                myViewport.Children.Add(modelsVisual);
                canvas1.Children.Add(myViewport);
    
                myViewport.Height = canvas1.Height;
                myViewport.Width = canvas1.Width;
                Canvas.SetTop(myViewport, 0);
                Canvas.SetLeft(myViewport, 0);
    
                sensor = KinectSensor.KinectSensors[0];
                sensor.DepthStream.Enable(DepthImageFormat.Resolution320x240Fps30);
                sensor.DepthFrameReady += DepthFrameReady;
                sensor.Start();
    
            }
    
            void DepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
            {
                DepthImageFrame imageFrame = e.OpenDepthImageFrame();
                if (imageFrame != null)
                {
                    short[] pixelData = new short[imageFrame.PixelDataLength];
                    imageFrame.CopyPixelDataTo(pixelData);
                    int temp = 0;
                    int i = 0;
                    for (int y = 0; y < 240; y += s)
                        for (int x = 0; x < 320; x += s)
                        {
                            temp = ((ushort)pixelData[x + y * 320]) >> 3;
                            ((TranslateTransform3D)points[i].Transform).OffsetZ = temp;
                            i++;
                                              
    
                        }                                                              
    
                }        
                           
            }
    
    
    
            private GeometryModel3D Triangle(double x, double y, double s)
            {
                Point3DCollection corners = new Point3DCollection();
                corners.Add(new Point3D(x, y, 0));
                corners.Add(new Point3D(x, y + s, 0));
                corners.Add(new Point3D(x + s, y + s, 0));
    
                Int32Collection Triangles = new Int32Collection();
                Triangles.Add(0);
                Triangles.Add(1);
                Triangles.Add(2);
    
                MeshGeometry3D tmesh = new MeshGeometry3D();
                tmesh.Positions = corners;
                tmesh.TriangleIndices = Triangles;
                tmesh.Normals.Add(new Vector3D(0, 0, -1));
    
                GeometryModel3D msheet = new GeometryModel3D();
                msheet.Geometry = tmesh;
                msheet.Material = new DiffuseMaterial(new SolidColorBrush(Colors.Red));
                return msheet;
            }
            
            private void canvas1_PreviewMouseLeftButtonDown(object sender, MouseButtonEventArgs e)
            {
    
                var clickedPoint = e.GetPosition((Canvas)sender);
                String xcoord = Convert.ToString(clickedPoint.X);
                String ycoord = Convert.ToString(clickedPoint.Y);
                textBox1.Text = (xcoord);
                textBox2.Text = (ycoord);
                            
            }    
            
                     
                            
                
    
             
        }  
    }
    


    Wednesday, June 13, 2012 3:13 PM
  • Hello Luke24dub,

    I'm really interested in that task. Do you've finished the program?

    Thursday, October 25, 2012 1:40 PM