none
how to calculate velocity of a point between two frames RRS feed

  • Question

  • Hello, 

    this problem is quite difficult for me to solve because i still have problem to manipulate frames specially in events so here is my question :

    I have a method in SkeletonDisplayManager Class called box_dimensions which determinates the dimensions of a bounding box of a skeleton !

            float minX = 2.2000f;
            float minY =  1.6000f;
            float minZ =  4;
            float maxX = -2.2000f;
            float maxY = -1.6000f;
            float maxZ = 0;
    
    public void box_dimensions(Skeleton sk)
            {
                
                foreach (Joint j in sk.Joints)
                {
    
                        float jx = j.Position.X;
                        float jy = j.Position.Y;
                        float jz = j.Position.Z;
    
                        // Populate min and max X.
                        if (jx < minX)
                        {
                            minX = jx;
                        }
                        if (jx > maxX)
                        {
                            maxX = jx;
                        }
    
                        // Populate min and max Y
                        if (jy < minY)
                        {
                            minY = jy;
                        }
                        if (jy > maxY)
                        {
                            maxY = jy;
                        }
    
                        // Populate min and max Z
                        if (jz < minZ)
                        {
                            minZ = jz;
                        }
                        if (jz > maxZ)
                        {
                            maxZ = jz;
                        }
                    }
    
                float W = maxX - minX  ;
                float H = maxY - minY ;
                float D = maxZ - minZ ;
                float WD = (float)Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2));
                
               
    
             }

    in the skeletonFramesreadyEvent args i called this method and then i will need to calculate time when finding H and WD 

    so the velocity is calculated as 

    (H1- H0) / T1-T0 

    where H0 and H1 are the value in the first and second frames 

    T0 and T1 are the times in the first and  second frames

    i really be thankfull for any help ! 


    DKF

    Thursday, May 28, 2015 1:15 PM

Answers

  • The velocity calculation you provided is correct: velocity = distance / time

    To calculate this in code for each joint of each skeleton you will need to do a few things:

    1. Create a class variable for previous time: DateTime previousTime.  Initialize to DateTime.Now in your constructor.
    2. Create a class variable for the current time: DateTime currentTime.  Initialize to DateTime.Now in your constructor.
    3. Create a class variable for the delta time, use either milliseconds or ticks: int deltaTimeMilliseconds
    4. Create a class variable for a dictionary of trackingID's to the previous frame's skeletons.  Dictionary<Int32, Skeleton> previousSkeletons = new Dictionary<Int32, Skeleton>();
    5. When you get a frame, find the change in distance between the current frame and the previous frame.
    6. When you get a frame, set the current time and previous time.

    Get the time:

    // When you get a frame: previousTime = currentTime; currentTime = DateTime.Now;

    deltaTimeMilliseconds = (currentTime - previousTime).Millisecond;


    Get the positions:

    // When you iterate over the skeletons.
    foreach (Skeleton skeleton in skeletons)
    {
        if (previousSkeletons.ContainsKey(
            skeleton.TrackingID)
        {
            // Find change in positions:
            Skeleton previous = previousSkeletons[skeleton.TrackingID];
            
            // Now compare skeleton.Joints to previous.Joints.
            // Update a local variable delatDistance.
            
            // Find rate from deltaDistance / deltaTime.
            // Update a class variable with this rate to use later.
    
            // Update previousSkeletons
            previousSkeletons[trackingID] = skeleton; 
        }
        else
        {
            // TrackingID has not been added to the dictionary.
            previousSkeletons.Add(skeleton.TrackingID, skeleton);
        }
    }
    

    Thursday, May 28, 2015 4:00 PM

All replies

  • The velocity calculation you provided is correct: velocity = distance / time

    To calculate this in code for each joint of each skeleton you will need to do a few things:

    1. Create a class variable for previous time: DateTime previousTime.  Initialize to DateTime.Now in your constructor.
    2. Create a class variable for the current time: DateTime currentTime.  Initialize to DateTime.Now in your constructor.
    3. Create a class variable for the delta time, use either milliseconds or ticks: int deltaTimeMilliseconds
    4. Create a class variable for a dictionary of trackingID's to the previous frame's skeletons.  Dictionary<Int32, Skeleton> previousSkeletons = new Dictionary<Int32, Skeleton>();
    5. When you get a frame, find the change in distance between the current frame and the previous frame.
    6. When you get a frame, set the current time and previous time.

    Get the time:

    // When you get a frame: previousTime = currentTime; currentTime = DateTime.Now;

    deltaTimeMilliseconds = (currentTime - previousTime).Millisecond;


    Get the positions:

    // When you iterate over the skeletons.
    foreach (Skeleton skeleton in skeletons)
    {
        if (previousSkeletons.ContainsKey(
            skeleton.TrackingID)
        {
            // Find change in positions:
            Skeleton previous = previousSkeletons[skeleton.TrackingID];
            
            // Now compare skeleton.Joints to previous.Joints.
            // Update a local variable delatDistance.
            
            // Find rate from deltaDistance / deltaTime.
            // Update a class variable with this rate to use later.
    
            // Update previousSkeletons
            previousSkeletons[trackingID] = skeleton; 
        }
        else
        {
            // TrackingID has not been added to the dictionary.
            previousSkeletons.Add(skeleton.TrackingID, skeleton);
        }
    }
    

    Thursday, May 28, 2015 4:00 PM
  • thanks ! i will try and tell you 

    DKF

    Friday, May 29, 2015 2:08 PM
  • Hello again,
    i tried to follow your instruction but a i've been confused when working in the class and the FramesreadyEvent. 

    i wrote another class that i called checkTime and i did this :

       class TimeManager
        {
            float minX = 2.2000f;
            float minY =  1.6000f;
            float minZ =  4;
            float maxX = -2.2000f;
            float maxY = -1.6000f;
            float maxZ = 0;
     
    
            DateTime previousTime = DateTime.Now;
            DateTime currentTime;
            private int deltaTimeMilliseconds;
            private float delatDistance;
    
            readonly Dictionary<Int32, Skeleton> previousSkeletons = new Dictionary<Int32, Skeleton>();
          
            public void CheckTime(SkeletonFrame frame)
            {
                if (frame != null )
                {
                    
                    currentTime = DateTime.Now ;
                  
                    deltaTimeMilliseconds = (currentTime - previousTime).Milliseconds ;
    
                    previousTime = currentTime; 
    
    
                    foreach (Skeleton skeleton in skeletons)
                    {
                      if (previousSkeletons.ContainsKey(skeleton.TrackingID))
                      {
                            // Find change in positions:
                             Skeleton previous = previousSkeletons[skeleton.TrackingID];
                                
                                  // Now compare skeleton.Joints to previous.Joints.
                               if (box_dimensions(skeleton) != box_dimensions(previous))
    	                      {
    		                 // Update a local variable delatDistance.
                                   delatDistance = box_dimensions(skeleton) - box_dimensions(previous) ;
                            // Find rate from deltaDistance / deltaTime.
                                   float Vh = delatDistance/deltaTimeMilliseconds ;
    
    
    	                     }
              
                        previousSkeletons[trackingID] = skeleton; 
                       }
                   else
                      previousSkeletons.Add(skeleton.TrackingID, skeleton); // TrackingID has not been added to the dictionary.
                   }
                
                 }
    
           }
    }

    skeletons in unknown , do i have to used it in the EventArgs directly  ? 

    also i used box dimension_methods to find H and WD but it can give me only one variable as return so how can i do to get both out ? 

     
             public float box_dimensions(Skeleton sk)
            {
                
                foreach (Joint j in sk.Joints)
                {
    
                        float jx = j.Position.X;
                        float jy = j.Position.Y;
                        float jz = j.Position.Z;
    
                        // Populate min and max X.
                        if (jx < minX)
                        {
                            minX = jx;
                        }
                        if (jx > maxX)
                        {
                            maxX = jx;
                        }
    
                        // Populate min and max Y
                        if (jy < minY)
                        {
                            minY = jy;
                        }
                        if (jy > maxY)
                        {
                            maxY = jy;
                        }
    
                        // Populate min and max Z
                        if (jz < minZ)
                        {
                            minZ = jz;
                        }
                        if (jz > maxZ)
                        {
                            maxZ = jz;
                        }
                    }
    
                float W = maxX - minX  ;
                float H = maxY - minY ;
                float D = maxZ - minZ ;
                float WD = (float)Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2));
                return H ;
           
    
             }
    

    you really helped me in this discussion so i hope that you continue with me until finding solution ..

    i will be gratefull

    thanks again !


    DKF

    Saturday, May 30, 2015 10:02 AM
  • Hello again ! 

    welle i tried another way but i don't know if the idea is correct .. 

    i tried to follow the the contextTracker Class in the Kinect Toolbox so i created two classes : ContextBox and ContextTrackerBox :

    public class ContextBox
        {
            public int Time
            {
                get;
                set;
            }
    
            public float H_Box
            {
                get;
                set;
    
            }
    
    
            public float WD_Box
            {
                get;
                set;
    
            }
    
        }
    
    
    public class ContextTrackerBox
        {
            readonly Dictionary<int, List<ContextBox>> boundingSkeletons = new Dictionary<int, List<ContextBox>>();
    //set threshold to detect fall and max and min distances
            readonly int framesBuffer;
            float minX = 2.2000f;
            float minY = 1.6000f;
            float minZ = 4;
            float maxX = -2.2000f;
            float maxY = -1.6000f;
            float maxZ = 0;
            public float Threshold_Vh { get; set; }
            public float Threshold_Vwd { get; set; }
            public float Threshold_Vih { get; set; }
           
     public ContextTrackerBox(int framesBuffer = 90, float threshold_Vh = 1.18f, float threshold_Vwd = 1.2f, float threshold_Vih = 0.5f )
            {
                 this.framesBuffer  = framesBuffer;
                 Threshold_Vh = threshold_Vh;
                 Threshold_Vih = threshold_Vih;
                 Threshold_Vwd = threshold_Vwd;
            }
    
            public void Add(float h_box, float wd_box, int trackingID)
            {
                if (!boundingSkeletons.ContainsKey(trackingID))
                {
                    boundingSkeletons.Add(trackingID, new List<ContextBox>());
                }
    
                boundingSkeletons[trackingID].Add(new ContextBox() { H_Box = h_box, WD_Box = wd_box, Time = DateTime.Now.Millisecond});
    
                if (boundingSkeletons[trackingID].Count > framesBuffer)  // 90 = 3 seconds then start flashing memory 
                {
                    boundingSkeletons[trackingID].RemoveAt(0);
                }
            }
    
     // calculate H and WD everyTime we detect Skeleton and Set time       
     public void Add(Skeleton skeleton)
            {
                var trackingID = skeleton.TrackingId;
                
                if (skeleton.TrackingState == SkeletonTrackingState.Tracked)
                {
                    foreach (Joint j in skeleton.Joints)
                    {
    
                        float jx = j.Position.X;
                        float jy = j.Position.Y;
                        float jz = j.Position.Z;
    
                        // Populate min and max X.
                        if (jx < minX)
                        {
                            minX = jx;
                        }
                        if (jx > maxX)
                        {
                            maxX = jx;
                        }
    
                        // Populate min and max Y
                        if (jy < minY)
                        {
                            minY = jy;
                        }
                        if (jy > maxY)
                        {
                            maxY = jy;
                        }
    
                        // Populate min and max Z
                        if (jz < minZ)
                        {
                            minZ = jz;
                        }
                        if (jz > maxZ)
                        {
                            maxZ = jz;
                        }
                    }
    
                    float W = maxX - minX;
                    float H = maxY - minY;
                    float D = maxZ - minZ;
                    float WD = (float)Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2));
                    Add(H, WD, trackingID);
                }
    
            }
    
          
    //  add skeletonBox in the boundingSkeletons dictionnary 
      public bool CheckForFall(int trackingID)
            {
    
                List<ContextBox> currentBox = boundingSkeletons[trackingID];
                if (currentBox.Count < 2)  // calculate Time only if we have two frames in buffer where there is a skeleton already known 
                    return false;
    
                float previousHeight = currentBox[currentBox.Count - 2].H_Box;
                float currentHeight = currentBox[currentBox.Count - 1].H_Box;
                float previousDiag = currentBox[currentBox.Count - 2].WD_Box;
                float currentDiag = currentBox[currentBox.Count - 1].WD_Box;
                int previousTime = currentBox[currentBox.Count - 2].Time;
                int currentTime = currentBox[currentBox.Count - 1].Time;
                var currentSpeed_Vh = (currentHeight - previousHeight) / ((currentTime - previousTime));
                var currentSpeed_Vwd = (currentDiag - previousDiag) / ((currentTime - previousTime));
                if ((currentSpeed_Vh > Threshold_Vh)&&(currentSpeed_Vwd > Threshold_Vwd ))
                    return false;
    
                return true;
            }
    
    
    
    
    
    
    
    
    
    
    
        }

    in the FramesReadyEvent args i did :

     void newSensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
            {
               
    
                    using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
                    {
    
                        if (skeletonFrame == null)
                            return;
                        skeletonFrame.GetSkeletonsT(ref skeletons);
    
                        if (skeletons.All(s => s.TrackingState == SkeletonTrackingState.NotTracked))
                        {
                            txtP.Text = "No Person detected";
                            return;
                        }
                        txtP.Text = "A person is detected";
                        // Action To do if a skeleton is detected 
    
    
                        skeletonDisplayManager.Draw(skeletons);
                        ThreedBoundingBox(skeletonFrame);
    
                        
                    
    
                }
            }
    
    
    

    with ThreedBoundingBox method :

    private void ThreedBoundingBox(SkeletonFrame sframe)
            {
               
                
                foreach (var sk in sframe.GetSkeletons() )
                {
                    if (sk.TrackingState != SkeletonTrackingState.Tracked)
                        continue;
                       contextTrackerBox.Add(sk);
                       contextTrackerBox.CheckForFall(sk.TrackingId);
                      
                
                
                }
            } 
    

    I did not test my program yet cause i have no Kinect actually.. but syntaxing there is no error..

    do you think that the logic of what i did is right ?  the work i followed was done to record and replay frames to detect gesture but i want a real time application and i think that the principle to calculate speed and time is the same .. 

    thanks for giving you opinion 

    Regards 


    DKF

    Sunday, May 31, 2015 3:41 PM
  • The code looks like it would work, but there might be some issues with garbage collection in the dictionary:

    readonly Dictionary<int, List<ContextBox>> boundingSkeletons = new Dictionary<int, List<ContextBox>>();

    Since you are checking for a fall only when you get a frame, you can get away with a dictionary with a single ContextBox:

    readonly Dictionary<int, ContextBox> boundingSkeletons = new Dictionary<int, ContextBox>();

    Whenever you get a frame, you can check if you have a non-null ContextBox, if the ContextBox is non-null, you can do your velocity check on the existing ContextBox and then overwrite.  When you overwrite, you can improve space complexity even more by using the existing ContextBox memory instead of creating a new ContextBox:

    DateTime currentTime = DateTime.Now; ContextBox box = boundingSkeletons[trackingID]; if (box == NULL) { boundingSkeletons[trackingID] = new ContextBox( /* Insert relevant args here */ );

    return; } else { // Perform velocity calculation with current // skeleton frame and non-null cached box. // Since boundingSkeletons[trackingID] is non-null, // we can update in-place: boundingSkeletons[trackingID].Time = currentTime; // Similarly, update all other fields like this. }


    Monday, June 1, 2015 6:23 PM
  • Thanks again for your reply ! :)

    As i told you i followed the contextTacker in the toolbox but i will also check this method to improve performance! 

    Actually to see if i detect a fall or not i'm using the debug mode and  checking falldetection bool variable but this is not really a good idea so i want to display the text "fall detected" in main window when i get true ! 

    That's what i'm trying to do now... 

    someone told me about the MVVM but it seems a little bit complicated...  is that what i need ?  

    thank you !



    DKF

    Monday, June 1, 2015 8:47 PM
  • Good morning ,

    i tried to use a Listbox in my main window that i called "fall_detected" 

    and in the  ThreedBoundingBox method i added a dictionary called "fall" then i display the items of this dictionary  in the ListBox

     private void ThreedBoundingBox(SkeletonFrame sframe)
            {
                Dictionary<int, string> fall = new Dictionary<int, string>();
                
                foreach (var sk in sframe.GetSkeletons() )
                {
                    if (sk.TrackingState != SkeletonTrackingState.Tracked)
                        continue;
                       contextTrackerBox.Add(sk);
                       contextTrackerBox.CheckForFall(sk.TrackingId);
                       fall.Add(sk.TrackingId, contextTrackerBox.CheckForFall(sk.TrackingId) ? "fall" : "Nothing");
                       fall_detected.ItemsSource = fall ;
                           
                       
                }
            }

    However i get a wired result which looks like reversed string: 

    have you any idea what's is this due to ? 


    DKF

    Tuesday, June 2, 2015 9:18 AM
  • What is the type of fall_detected?

    What you are seeing is fall_detected trying to get a text representation of the memory of the fall dictionary.  fall_detected is likely expecting some other data type, probably just a string.

    Tuesday, June 2, 2015 7:21 PM
  • Hello 

    fall_detected is the name of my ListBox in the mainWindow !

    and what i did is to add the result of CheckForFall method to that ListBox.. the values of the dictionary "fall" are already strings  so... ?


    DKF

    Wednesday, June 3, 2015 6:08 AM
  • A ListBox is expecting a List<String>, if you want to bind a dictionary, you will have to change your XAML to look something like this:

    <ListBox x:Name="fall_detected">
        <ListBox.ItemTemplate>
            <DataTemplate>
                <StackPanel Orientation="Horizontal">
                    <TextBlock Text="{Binding Key}"/>
                    <TextBlock Text="{Binding Value}"/>
                </StackPanel>
            </DataTemplate>
        </ListBox.ItemTemplate>
     </ListBox>
    Wednesday, June 3, 2015 6:30 AM