none
Problems for a beginner in building a simple Kinect V2 plugin for Unity3D RRS feed

  • Question

  • Hello, 

    I need face rotation in my Unity3D project, so, I'm trying to build a simple unity plugin, but I'm a beginner in that and I need help!

    I have created a native C++ that builds a dll file that I should wrap in a C# wrapper to Unity. 

    In that C++ code, I have this code snippet (taken from Kinect SDK):

    #if _MSC_VER // this is defined when compiling with Visual Studio
    #define EXPORT_API __declspec(dllexport) // Visual Studio needs annotating exported functions with this
    #else
    #define EXPORT_API // XCode does not need annotating exported functions, so define is empty
    #endif
    
    
    #include "stdafx.h"
    #include "resource.h"
    #include <iostream>
    
    extern "C"
    {
    	HRESULT EXPORT_API InitializeDefaultSensor()
    	{
    		IKinectSensor*          m_pKinectSensor;
    		IFaceFrameSource*		m_pFaceFrameSources[BODY_COUNT];
    		IFaceFrameReader*		m_pFaceFrameReaders[BODY_COUNT];
    		IColorFrameReader*      m_pColorFrameReader;
    		IBodyFrameReader*       m_pBodyFrameReader;
    
    		static const DWORD c_FaceFrameFeatures =
    			FaceFrameFeatures::FaceFrameFeatures_BoundingBoxInColorSpace
    			| FaceFrameFeatures::FaceFrameFeatures_PointsInColorSpace
    			| FaceFrameFeatures::FaceFrameFeatures_RotationOrientation
    			| FaceFrameFeatures::FaceFrameFeatures_Happy
    			| FaceFrameFeatures::FaceFrameFeatures_RightEyeClosed
    			| FaceFrameFeatures::FaceFrameFeatures_LeftEyeClosed
    			| FaceFrameFeatures::FaceFrameFeatures_MouthOpen
    			| FaceFrameFeatures::FaceFrameFeatures_MouthMoved
    			| FaceFrameFeatures::FaceFrameFeatures_LookingAway
    			| FaceFrameFeatures::FaceFrameFeatures_Glasses
    			| FaceFrameFeatures::FaceFrameFeatures_FaceEngagement;
    
    		
    		HRESULT hr;
    
    		hr = GetDefaultKinectSensor(&m_pKinectSensor);
    		if (FAILED(hr))
    		{
    			return hr;
    		}
    
    		if (m_pKinectSensor)
    		{
    			// Initialize Kinect and get color, body and face readers
    			IColorFrameSource* pColorFrameSource = nullptr;
    			IBodyFrameSource* pBodyFrameSource = nullptr;
    
    			hr = m_pKinectSensor->Open();
    
    			if (SUCCEEDED(hr))
    			{
    				std::cout << "sensor opened" << std::endl;
    				hr = m_pKinectSensor->get_ColorFrameSource(&pColorFrameSource);
    			}
    
    			if (SUCCEEDED(hr))
    			{
    				std::cout << "color frame source got" << std::endl;
    				hr = pColorFrameSource->OpenReader(&m_pColorFrameReader);
    			}
    
    			if (SUCCEEDED(hr))
    			{
    				std::cout << "color frame source reader opened" << std::endl;
    				hr = m_pKinectSensor->get_BodyFrameSource(&pBodyFrameSource);
    			}
    
    			if (SUCCEEDED(hr))
    			{
    				std::cout << "body frame source got" << std::endl;
    				hr = pBodyFrameSource->OpenReader(&m_pBodyFrameReader);
    			}
    
    			if (SUCCEEDED(hr))
    			{
    				std::cout << "body frame source reader opened" << std::endl;
    				// create a face frame source + reader to track each body in the fov
    				for (int i = 0; i < BODY_COUNT; i++)
    				{
    					if (SUCCEEDED(hr))
    					{
    
    						// create the face frame source by specifying the required face frame features
    						// the next statement return failed HRESULT if you wrap it as unity plugin, but it works if in a console windows app!
    						hr = CreateFaceFrameSource(m_pKinectSensor, 0, c_FaceFrameFeatures, &m_pFaceFrameSources[i]);
    					}
    					if (SUCCEEDED(hr))
    					{
    						// open the corresponding reader
    						hr = m_pFaceFrameSources[i]->OpenReader(&m_pFaceFrameReaders[i]);
    					}
    				}
    			}
    
    			SafeRelease(pColorFrameSource);
    			SafeRelease(pBodyFrameSource);
    		}
    
    		if (!m_pKinectSensor || FAILED(hr))
    		{
    			// SetStatusMessage(L"No ready Kinect found!", 10000, true);
    			return E_FAIL;
    		}
    
    		return hr;
    	}
    }


    The problem is: this line of code from above:

    hr = CreateFaceFrameSource(m_pKinectSensor, 0, c_FaceFrameFeatures, &m_pFaceFrameSources[i]);

    returns FAILED(hr), but the strange thing is that if you build the solution as a Console application, it works!

    maybe I'm not good at building a plugin, but I really need help.

    Thanks..


    Monday, September 8, 2014 8:40 AM

All replies

  • You may have an issue with marshalling from the .Net api. Are you passing in a valid tracked user id?

    Carmine Sirignano - MSFT

    Tuesday, September 9, 2014 7:15 PM
  • Thanks for the answer,

    I have learned (and copied) the code from the sample (FaceBasics-D2D) and that portion of code is from the "InitializeDefaultSensor" and it doesn't need or use the (Tracked user id) as I understood (I'm a beginner as I said).

    The strange thing is that the code runs and returns OK when I compile it as a Console application, but when I export it as a DLL and wrap it, the "hr" returned from this certain line of code is FAILD:

    hr = CreateFaceFrameSource(m_pKinectSensor, 0, c_FaceFrameFeatures, &m_pFaceFrameSources[i]);
    m_pKinectSensor // Initialized, OK
    c_FaceFrameFeatures // is const struct
    m_pFaceFrameSources // array of size: BODY_COUNT and they are null

    That's what confuses me!

    I'm wondering if you know if a unity plugin that supports FaceRotation will be released soon or not?

    Thanks,

    Wednesday, September 10, 2014 11:59 AM
  • CreateFaceFrameSource requires a valid tracked user id - the value 0, is the tracked id from the Body, where 0 is invalid.


    Carmine Sirignano - MSFT

    Wednesday, September 10, 2014 5:07 PM