locked
MIDL 32 bit vs 64 bit IDL issue RRS feed

  • Question

  • For nearly 23 years,  in our RPC client/server system,we had a SDK specific function IDL defined as:

      error_status_t WcReadLine([in] TWildcatContextHandle wch,
                                [in] DWORD h,
                                [out, ref, size_is(*bufsize)] byte *buffer,
                                [in] DWORD *bufsize);

    C/C++ clients have a wcSDK header file and its prototype for this is:

    BOOL APIENTRY WcReadLine(WCHANDLE h, LPVOID buffer, DWORD bufsize)
    {
      InitContext(FALSE);  // make sure we connected to RPC server
      DWORD status = WcsWcReadLine(wch, (unsigned long)h, (byte *)buffer, &bufsize);
      if (status) {
        SetLastError(status);
        return FALSE;
      }
      return TRUE;
    }


    This has always worked and we have thousands of client apps and tools that use this successfully without a problem.

    When I compile under 64 bit, I ran into a problem where the status return by the marshalling code is 1783 "The stub received bad data."

    What I had to do was change the IDL to add the out and ref attributes:

      error_status_t WcReadLine([in] TWildcatContextHandle wch,
                                [in] DWORD h,
                                [out, ref, size_is(*bufsize)] byte *buffer,
                                [in, out, ref] DWORD *bufsize);

    Then it works, 64 bit client to 64 bit server,  however,  the function is no longer compatible with the 32 bit server or when compiled under 32 bit not compatible with the 64 bit server.  I have not totally checked all our 150+ wcSDK functions for similar issues, but overall, I don't have any issue with any other wcSDK function in this 32/64 bit mixed RPC client/server mode which is what is selling us the idea to proceed with the 64 bit migration -- the investment of thousands of apps by hundreds of developers with their 32 bit appsl is not lost with a 64 bit RPC server.

    What do you suggest?  I need to maintain wcSDK compatibility.  I could recompile the interface WCSRV2.DLL that all clients use, but I was wondering if I could do something here to maintain the same interface.  Could this be a MIDL thing with a special switch?

    Thanks



    Hector Santos, CTO Santronics Software, Inc. http://www.santronics.com



    Sunday, March 24, 2019 3:00 PM

Answers

  • You know, sometimes when work out the basic issue in your mind, write it down, like I did here, quite often the solution comes like it did now. :)   

    What the 64 bit compile showed is that we really had coding bug in server side stub that was writing back to the bufsize but the 32 bit marshalling code did not complain about it.  The 64 bit code had a problem with it.  The original code:

    /* [fault_status][comm_status] */ error_status_t WcsWcReadLine(
        /* [in] */ TWildcatContextHandle wch,
        /* [in] */ unsigned long h,
        /* [size_is][ref][out] */ byte __RPC_FAR *buffer,
        /* [in] */ unsigned long __RPC_FAR *bufsize)
    {
      TClientContext *cc = GetClientContext(wch);
      TFile *f = cc->HandleTable.GetHandlePtr(h);
      if (!f) {
        return ERROR_INVALID_HANDLE;
      }
      if (!f->ReadLine(buffer, *bufsize)) {
        return GetLastError();
      }
      return 0;
    }
    
    

    The fix was simple:

      // 454.8, fixes 64 bit marshal code corruption
      DWORD requested = *bufsize;
      if (!f->ReadLine(buffer, requested)) {
        return GetLastError();
      }

    closing this :)


    Hector Santos, CTO Santronics Software, Inc. http://www.santronics.com


    Sunday, March 24, 2019 3:34 PM