none
Recording Kinect v2 footage and playback RRS feed

  • Question

  • Hi,

    Been trying to work out how to record some clips of footage using the Kinect v2 and then play back the footage.

    I am using the body basics samples and have found the following functions to record and playback, the record function works and records footage but makes the live feed freeze whilst recording and the playback function freezes the live footage and then seems to show a couple of frames of data before the live body basics feed resumes. The functions I am using are below. I have simply added a couple of buttons to the body basics sample, record and playback and then call the functions in the event handlers, saving to a .xef file.

    public static void RecordClip(string filePath, TimeSpan duration)
            {
                using (KStudioClient client = KStudio.CreateClient())
                {
                    client.ConnectToService();

                    KStudioEventStreamSelectorCollection streamCollection = new KStudioEventStreamSelectorCollection();
                    streamCollection.Add(KStudioEventStreamDataTypeIds.Ir);
                    streamCollection.Add(KStudioEventStreamDataTypeIds.Depth);
                    streamCollection.Add(KStudioEventStreamDataTypeIds.Body);
                    streamCollection.Add(KStudioEventStreamDataTypeIds.BodyIndex);

                    using (KStudioRecording recording = client.CreateRecording(filePath, streamCollection))
                    {
                        recording.StartTimed(duration);
                        while (recording.State == KStudioRecordingState.Recording)
                        {
                            System.Threading.Thread.Sleep(500);
                        }

                        if (recording.State == KStudioRecordingState.Error)
                        {
                            throw new InvalidOperationException("Error:Recording failed!");
                        }
                    }

                    client.DisconnectFromService();
                }
            }

            public static void PlaybackClip(string filePath, uint loopCount)
            {
                using (KStudioClient client = KStudio.CreateClient())
                {
                    client.ConnectToService();

                    using (KStudioPlayback playback = client.CreatePlayback(filePath))
                    {
                        playback.LoopCount = loopCount;
                        playback.Start();

                        while (playback.State == KStudioPlaybackState.Playing)
                        {
                            System.Threading.Thread.Sleep(500);
                        }

                        if (playback.State == KStudioPlaybackState.Error)
                        {
                            throw new InvalidOperationException("Error: Playback failed!");
                        }
                    }

                    client.DisconnectFromService();
                }
            }

    I can only find minimal documentation on the KStudioClient class, so help would be massively appreciated.

    Friday, October 3, 2014 9:17 AM

Answers

  • The api's work the same as if you are using the KinectStudio(KStudio) app. Take the scenario when you are playing back a clip from KStudio. This will interrupt the live feed to playback then recorded data, since it is taking over from the driver. If you playback the clip in KStudio you should get the same behavior.

    That said Body data is stateless. Even though you may have recorded 500seconds, the amount of body data is generated based on the Depth/IR information in the clip. You need to ensure the clip recording has the frames to acquire a lock-on if you expect to have some consistency during each playback. Even then, you will have some variance given the state of the system you are using will not be the same during each playback.


    Carmine Sirignano - MSFT

    Monday, October 6, 2014 6:18 PM
  • I have posted a basic record/playback sample here, which shows how to use the Kinect tooling APIs: https://github.com/angelaHillier/RecordAndPlaybackBasics-WPF.git
    Tuesday, June 30, 2015 6:29 AM

All replies

  • The api's work the same as if you are using the KinectStudio(KStudio) app. Take the scenario when you are playing back a clip from KStudio. This will interrupt the live feed to playback then recorded data, since it is taking over from the driver. If you playback the clip in KStudio you should get the same behavior.

    That said Body data is stateless. Even though you may have recorded 500seconds, the amount of body data is generated based on the Depth/IR information in the clip. You need to ensure the clip recording has the frames to acquire a lock-on if you expect to have some consistency during each playback. Even then, you will have some variance given the state of the system you are using will not be the same during each playback.


    Carmine Sirignano - MSFT

    Monday, October 6, 2014 6:18 PM
  • Hello,

    The above code will work great when running from a separate command line tool or application, but if you were to add this code directly to one of the WPF samples, then you will fail to see anything visualized during record/playback because the main thread will be frozen. Instead, you will want to call the record and playback functions asynchronously, by adding them as background tasks to the Dispatcher. Here is an example of how you can do this in WPF:

     /// <summary> Delegate for placing a job with no arguments onto the Dispatcher </summary>
     private delegate void NoArgDelegate();
    
     /// <summary> Delegate for placing a job with a single string argument onto the Dispatcher </summary>
     /// <param name="arg">string argument</param>
     private delegate void OneArgDelegate(string arg);
    
     ///
     /// Code to initialize MainWindow, KStudioClient, etc. should go here
     ///       
    
     /// <summary>
     /// Handles the user clicking on the Record button
     /// </summary>
     private void RecordButton_Click(object sender, RoutedEventArgs e)
     {
         this.isRecording = true;
         this.UpdateButtonState();
    
         // Start running the recording asynchronously
         OneArgDelegate recording = new OneArgDelegate(this.RecordClip);
         recording.BeginInvoke(@"C:\temp\kinectRecording0.xef", null, null);
     }
    
     /// <summary>
     /// Handles the user clicking on the Play button
     /// </summary>
     private void PlayButton_Click(object sender, RoutedEventArgs e)
     {
         this.isPlaying = true;
         this.UpdateButtonState();
         // Start running the playback asynchronously
         OneArgDelegate playback = new OneArgDelegate(this.PlaybackClip);
         playback.BeginInvoke(@"C:\temp\kinectRecording0.xef", null, null);
     }
    
     /// <summary>
     /// Records a new .xef file
     /// </summary>
     private void RecordClip(string filePath)
     {
         // Specify which streams should be recorded
         KStudioEventStreamSelectorCollection streamCollection = new KStudioEventStreamSelectorCollection();
                     streamCollection.Add(KStudioEventStreamDataTypeIds.Ir);
                     streamCollection.Add(KStudioEventStreamDataTypeIds.Depth);
                     streamCollection.Add(KStudioEventStreamDataTypeIds.Body);
                     streamCollection.Add(KStudioEventStreamDataTypeIds.BodyIndex);
    
         // Create the recording object
         using (KStudioRecording recording = this.client.CreateRecording(filePath, streamCollection))
         {
             recording.StartTimed(this.duration);
             while (recording.State == KStudioRecordingState.Recording)
             {
                 Thread.Sleep(500);
             }
         }
    
         // Update UI after the background recording task has completed
         this.isRecording = false;
         this.Dispatcher.BeginInvoke(new NoArgDelegate(UpdateButtonState));
     }
    
     /// <summary>
     /// Plays back a .xef file to the Kinect sensor
     /// </summary>
     private void PlaybackClip(string filePath)
     {
        // Create the playback object
        using (KStudioPlayback playback = client.CreatePlayback(filePath))
        {
            playback.LoopCount = this.loopCount;
            playback.Start();
    
            while (playback.State == KStudioPlaybackState.Playing)
            {
                Thread.Sleep(500);
            }
        }
    
        // Update the UI after the background playback task has completed
        this.isPlaying = false;
        this.Dispatcher.BeginInvoke(new NoArgDelegate(UpdateButtonState));
     }


    Monday, November 3, 2014 3:58 AM
  • hi, Kev3

    I am  so confused cause I have been learning Kinect for a limited period. Do you know how to  call the functions in the event handlers now?

     
      private void record_Click(object sender, RoutedEventArgs e)
            {
                   //What should I do here?
            }
    
      private void replay_Click(object sender, RoutedEventArgs e)
            {
              //What should I do here?
    
            }

    Wednesday, March 25, 2015 1:49 PM
  • hi, angela

    couly you tell me what the content of the function UpdateButtonState()?

    How to declare the this.loopCount\this.duration?

    Thank you a lot 


    • Edited by oniya Thursday, March 26, 2015 10:57 AM
    Thursday, March 26, 2015 10:40 AM
  • I have posted a basic record/playback sample here, which shows how to use the Kinect tooling APIs: https://github.com/angelaHillier/RecordAndPlaybackBasics-WPF.git
    Tuesday, June 30, 2015 6:29 AM