none
.net 4 pinvoke problem RRS feed

  • Question

  • Hello everyone.
    I have winapi function SetWindowsHookEx declared this way:
    public enum HookType : int
    {
    WH_JOURNALRECORD = 0,
    WH_JOURNALPLAYBACK = 1,
    WH_KEYBOARD = 2,
    WH_GETMESSAGE = 3,
    WH_CALLWNDPROC = 4,
    WH_CBT = 5,
    WH_SYSMSGFILTER = 6,
    WH_MOUSE = 7,
    WH_HARDWARE = 8,
    WH_DEBUG = 9,
    WH_SHELL = 10,
    WH_FOREGROUNDIDLE = 11,
    WH_CALLWNDPROCRET = 12,
    WH_KEYBOARD_LL = 13,
    WH_MOUSE_LL = 14
    }


    // hook method called by system
    public delegate int HookProc(int code, IntPtr wParam, IntPtr lParam);


    [DllImport("user32.dll", SetLastError=true)]
    public static extern IntPtr SetWindowsHookEx(
    HookType code, HookProc func, IntPtr instance, int threadID);

    And i call to it
    			Module[] list = System.Reflection.Assembly.GetExecutingAssembly().GetModules();
    System.Diagnostics.Debug.Assert(list != null && list.Length > 0);

    // install system-wide hook
    _HookHandle = HookAPI.SetWindowsHookEx(
    _HookType,
    _HookFunction, 
    Marshal.GetHINSTANCE(list[0]), 
    0);
      var le = Marshal.GetLastWin32Error();

    The problem is that it works fine in .net 3.5 but do not in .net 4.0.
    Also Marshal.GetLastWin32Error() returns zero and no exception is thrown.
    I also tried <NetFx40_PInvokeStackResilience enabled="1"/> key but it do not help to solve the problem .

    Some update: In release build Marshal.GetLastWin32Error() returned 6(invalid handle).
    I think problem is that Marshal.GetHINSTANCE(list[0]) returns something different in .net 2.0 and 4.0.

    Further investigation: .net 4 has breaking change: it loads assembly without call to LoadLibrary, so there is no loaded module for referenced assembly. But Marshal.GetHINSTANCE(list[0]) still returns some value and it is not hInstance of Entry Assembly.

    Amazing! On 64bit windows and .net 4 it still works. And I see loaded modules. Does framework 4.0. loads assemblies in different way on 32bit and 64bit windows?
    Monday, November 22, 2010 6:19 PM

All replies