none
Upgraded to Kinect 1.8 Sdk and now getting an error where I wasnt using the 1.7 RRS feed

  • Question

  • Hi,

    I recently upgraded my Kinect SDK from the 1.7 to the 1.8.  I had an extension method for the byte[] of the color image frame as follows:

            public static byte[] ToJpegBytes(this byte[] data, int compressionRate)
            {
                // On the event of no data being passed, return null;
                if (data.IsNullOrEmpty())
                    return null;
                TypeConverter tc = TypeDescriptor.GetConverter(typeof(Bitmap));
    
                using (Bitmap bmp = (Bitmap)tc.ConvertFrom(data))
                {
                    System.Drawing.Imaging.ImageCodecInfo myImageCodecInfo;
                    System.Drawing.Imaging.Encoder myEncoder;
                    System.Drawing.Imaging.EncoderParameter myEncoderParameter;
                    System.Drawing.Imaging.EncoderParameters myEncoderParameters;
                    myImageCodecInfo = GetEncoderInfo("image/jpeg");
                    myEncoder = System.Drawing.Imaging.Encoder.Quality;
                    myEncoderParameters = new System.Drawing.Imaging.EncoderParameters(1);
    
                    // Save the bitmap as a JPEG file with quality level 25.
                    myEncoderParameter = new System.Drawing.Imaging.EncoderParameter(myEncoder, compressionRate);
                    myEncoderParameters.Param[0] = myEncoderParameter;
                    
                    using (var stream = new MemoryStream())
                    {
                        bmp.Save(stream, myImageCodecInfo, myEncoderParameters);
                        return stream.ToArray();
                    }
                }
            }

     

    This allows me to compress the image to JPEG format then display it later.  This now throws an invalid parameter error.  When I revert back to the 1.7 SDK this works again.

    Can someone tell me whats changed or what I'm doing wrong?

    Thanks in advance!

    Tuesday, October 22, 2013 10:07 PM

Answers

  • Hi Carmine,

    I tried with CopyPixelDataTo() method also.  Its hard to tell if the byte data is completely valid when the Convert takes place (there's something wrong with it as it throws the exception) but in the quick view it looks ok.

    For anyone else who was stuck on the same issue, here's a work around:

    WriteableBitmap bmp = new WriteableBitmap(640, 480, 96.0, 96.0, PixelFormats.Bgr32, null);
    bmp.WritePixels(new Int32Rect(0, 0, 640, 480), data, bmp.PixelWidth * sizeof(int), 0);
    var bitmapSource = new WriteableBitmap(bmp);
    
    var enc = new JpegBitmapEncoder();
    enc.QualityLevel = ratio;
    
    var bf = BitmapFrame.Create(bitmapSource); // will throw InvalidOperationException (without WriteableBitmap trick)
    enc.Frames.Add(bf);
    
    using (var ms = new MemoryStream())
    {
        enc.Save(ms);
        BitmapImage NewBitmap = new BitmapImage();
        NewBitmap.BeginInit();
        NewBitmap.StreamSource = new MemoryStream(ms.ToArray());
        NewBitmap.EndInit();
        return NewBitmap;   
    }

     Thanks for your help.
    • Marked as answer by rmccabe82 Wednesday, October 30, 2013 8:36 AM
    Wednesday, October 30, 2013 8:36 AM

All replies

  • There have been no changes to the SDK or format of the data in 1.8 that would change this behavior.

    What line of code is throwing the error? Given that the code you show has no Kinect specific API or data type, I would say it is how you are calling this that might be the issue. How are you creating the data that is passed to the function?


    Carmine Sirignano - MSFT


    Wednesday, October 23, 2013 12:24 AM
  • Hi Carmine,

    Thanks for getting back to me.  Here is how I call the extension method above.  This code lies in the AllFramesReady event handler method:

     
                         int compressionrate = (int)sldCompressionRate.Value;
    
                            // Return null if we're passed an empty frame.
                            if (colorFrame.IsNullOrEmpty())
                                return null;
    
                            // Get byte array of data.
                            var data = new byte[colorFrame.PixelDataLength];
                            colorFrame.CopyPixelDataTo(data);
    
                            imgOne.Source = data.ToJpegBytes(compressionrate).ToBitmapSource();

    Thanks,

    Rob 



    • Edited by rmccabe82 Wednesday, October 23, 2013 7:50 AM
    Wednesday, October 23, 2013 7:49 AM
  • I am not familiar with that code syntax for ".ToJpegBytes" and we don't use that in our samples. Does the ColorBasics-WPF sample work for you? Also are you calling .Dispose on your "colorFrame" or is there a using statement when you call OpenColorImageFrame?

    using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
    {
        if (colorFrame != null)
        {
            // Copy the pixel data from the image to a temporary array
            colorFrame.CopyPixelDataTo(this.colorPixels);
    
            // Write the pixel data into our bitmap
            this.colorBitmap.WritePixels(
                new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
                this.colorPixels,
                this.colorBitmap.PixelWidth * sizeof(int),
                0);
        }
    }
    


    Carmine Sirignano - MSFT

    Wednesday, October 23, 2013 8:26 PM
  • Hi Carmine,

    .ToJpegBytes() is my own method (shown in my first post on this thread).  I use it to convert the full Kinect color bytes to a compressed version of the bytes.  I do this as I write the data to disk and the compressed data is smaller in size (on disk) than the full color image.

    Your code works for me in the above example, but that wont be the solution for me.  I need to convert the image coming from the color stream to a compressed JPEG.  The code conversion worked up until I installed the new sdk last week.  And works if I revert back to the 1.7 sdk.  Can you offer an alternative way to convert the color frame bytes to a jpeg that works with the 1.8 sdk?

    Many thanks!


    • Edited by rmccabe82 Thursday, October 24, 2013 1:15 PM
    Thursday, October 24, 2013 1:13 PM
  • What line of code is throwing the error and what is the actual error you are getting? Did you use a "using" statement around the use of the colorFrame to ensure you are releasing the frame?

    You could use the System.Windows.Media.Imaging.BitmapEncoder
    http://msdn.microsoft.com/en-us/library/system.windows.media.imaging.bitmapencoder.aspx

    Are you also using the default color stream(BGRA)? Keep in mind that there is also the IR and YUV formats if you happen to have code that flips between these, you have to ensure you are reading the right data types.


    Carmine Sirignano - MSFT

    Thursday, October 24, 2013 5:53 PM
  • Hi,

    I initialise my Kinect color stream as follows:

    sensor.AllFramesReady += new EventHandler<AllFramesReadyEventArgs>(sensor_AllFramesReady);
    
    sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
                
    try
    {
       sensor.Start();
       Message = "Kinect connected";
    }
    catch (Exception ex)
    {
       Message =  ex.Message;
       DeInitSensor(sensor);
    }

    And in the AllFramesReady event here is my code:

    using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
    {
        if (colorFrame != null)
        {
            var bytes = colorFrame.GetRawPixelData();
            var jpegBytes = bytes.ToJpegBytes(50);
        }
    }

    The error happens when I hit the following line within the ToJpegBytes() extension method called in the sample above:

    using (Bitmap bmp = (Bitmap)tc.ConvertFrom(data))

    The error is an argument exception of "Parameter is not valid".  Here is the exact error:

      System.ArgumentException was unhandled
      HResult=-2147024809
      Message=Parameter is not valid.
      Source=System.Drawing
      StackTrace:
    at System.Drawing.Image.FromStream(Stream stream, Boolean useEmbeddedColorManagement, Boolean validateImageData) 
    at System.Drawing.ImageConverter.ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, Object value)
    at System.ComponentModel.TypeConverter.ConvertFrom(Object value) 
    at System.ByteExtension.ToJpegBytes(Byte[] data, Int32 compressionRate) 
    
    Thanks
    Friday, October 25, 2013 9:09 AM
  • Just tried the JpegBitmapEncoder as you suggested and get an error as well.  Here is my code for the all frames ready handler:

    void sensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
            {
                using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
                {
                    if (colorFrame != null)
                    {
                        var bytes = colorFrame.GetRawPixelData();        
                        byte[] jpegBytes = new byte[0];
    
                        using (var ms = new MemoryStream(bytes))
                        {
                            JpegBitmapEncoder encoder = new JpegBitmapEncoder();
                            encoder.QualityLevel = 50;
                            encoder.Frames.Add(BitmapFrame.Create(ms));
                            using (MemoryStream stream = new MemoryStream())
                            {
                                encoder.Save(stream);
                                jpegBytes = stream.ToArray();
                            }
                        }
                    }
                }

    In this example, the error happens at line:

    encoder.Frames.Add(BitmapFrame.Create(ms));
    With error "No imaging component suitable to complete this operation was found.":

    System.NotSupportedException was unhandled
    HResult=-2146233067
    Message=No imaging component suitable to complete this operation was found.
    Source=PresentationCore
    StackTrace:
           at System.Windows.Media.Imaging.BitmapDecoder.SetupDecoderFromUriOrStream(Uri uri, Stream stream, BitmapCacheOption cacheOption, Guid& clsId, Boolean& isOriginalWritable, Stream& uriStream, UnmanagedMemoryStream& unmanagedMemoryStream, SafeFileHandle& safeFilehandle)
    
           at System.Windows.Media.Imaging.BitmapDecoder.CreateFromUriOrStream(Uri baseUri, Uri uri, Stream stream, BitmapCreateOptions createOptions, BitmapCacheOption cacheOption, RequestCachePolicy uriCachePolicy, Boolean insertInDecoderCache)
    
           at System.Windows.Media.Imaging.BitmapDecoder.Create(Stream bitmapStream, BitmapCreateOptions createOptions, BitmapCacheOption cacheOption)
    
           at System.Windows.Media.Imaging.BitmapFrame.CreateFromUriOrStream(Uri baseUri, Uri uri, Stream stream, BitmapCreateOptions createOptions, BitmapCacheOption cacheOption, RequestCachePolicy uriCachePolicy)
    
           at System.Windows.Media.Imaging.BitmapFrame.Create(Stream bitmapStream)
    
           at WordSign.Conversion.MainWindow.sensor_AllFramesReady(Object sender, AllFramesReadyEventArgs e)
    

    Can you let me know what Im doing wrong?  This seems like a fairly straight forward piece of code.

    Thanks!


    Friday, October 25, 2013 9:25 AM
  • I can't speak to specifics of .Net functionality so you may need to engage with the .Net framework team to know for sure. If you set a breakpoint in the function, is the byte data for the frame still valid when it goes to use it? Also does it fail on the first frame, or are there some frames it work and then fails?

    As far as the Kinect for Windows SDK goes, try using the CopyPixelDataTo function to ensure your application has a copy of the frame data.

    // define a class level variable
    private byte[] colorPixels;
    
    // Allocate space to put the pixels
    this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength];
    
    
    // event handler
    private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
    {
        using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
        {
            if (colorFrame != null)
            {
                // Copy the pixel data from the image to a temporary array
                colorFrame.CopyPixelDataTo(this.colorPixels);
    
                // do something here with this.colorPixels
            }
        }
    }

    It is a best practice to define the variable once since the frame sizes don't change and you are not recreating managed memory all the time.

    When using GetRawPixelData() method you have to ensure you are using the data as fast as possible since there is no guarantee the Kinect runtime will not reclaim the memory. There may be some timing differences from before that are affecting the release of the frame, which is why it worked before but not now. By the time the byte converter executes the data from the runtime may be reclaimed.

    Using the copy method, you shouldn't have this issue.


    Carmine Sirignano - MSFT




    Monday, October 28, 2013 5:11 PM
  • Hi Carmine,

    I tried with CopyPixelDataTo() method also.  Its hard to tell if the byte data is completely valid when the Convert takes place (there's something wrong with it as it throws the exception) but in the quick view it looks ok.

    For anyone else who was stuck on the same issue, here's a work around:

    WriteableBitmap bmp = new WriteableBitmap(640, 480, 96.0, 96.0, PixelFormats.Bgr32, null);
    bmp.WritePixels(new Int32Rect(0, 0, 640, 480), data, bmp.PixelWidth * sizeof(int), 0);
    var bitmapSource = new WriteableBitmap(bmp);
    
    var enc = new JpegBitmapEncoder();
    enc.QualityLevel = ratio;
    
    var bf = BitmapFrame.Create(bitmapSource); // will throw InvalidOperationException (without WriteableBitmap trick)
    enc.Frames.Add(bf);
    
    using (var ms = new MemoryStream())
    {
        enc.Save(ms);
        BitmapImage NewBitmap = new BitmapImage();
        NewBitmap.BeginInit();
        NewBitmap.StreamSource = new MemoryStream(ms.ToArray());
        NewBitmap.EndInit();
        return NewBitmap;   
    }

     Thanks for your help.
    • Marked as answer by rmccabe82 Wednesday, October 30, 2013 8:36 AM
    Wednesday, October 30, 2013 8:36 AM