I've posted this question on InstallShieds forums ( here ), but as of yet, have not gotten an answer from their community -- I was
hoping to have better luck here.
The issue I am having is that I am trying to invoke a method from the advapi32.dll from inside InstallShield's basic MSI. I successfully called GetUsernameA(...) from this routine -- so linkage is not the issue.
Rather, I am trying to call LsaOpenPolicy(...) and am getting an exception of "mismatched type": 0x80020005. I'm a bit stymied as to how to proceed -- I've tried a few variants of prototype and invocation, all to no avail.
My latest attempt defined the prototype as:
prototype INT Advapi32.LsaOpenPolicy(POINTER, POINTER, INT, POINTER);
The InstallShield call looks like this (including some setup -- all examples of C++ code zero's out the structure pointed to by the second parameter).
for i = 0 to 11
array(i) = 0;
array(0) = 24;
// current error is 80020005 type mismatch.
i = POLICY_CREATE_ACCOUNT | POLICY_LOOKUP_NAMES;
pArray = array;
pPolicy = NULL;
nvOSResult = LsaOpenPolicy(NULL, pArray, i, pPolicy);
Sprintf(errString, "0x%08x", Err.Number);
_Logger(hMSI, methodName, "LsaOpenPolicy Exception "+errString, INFORMATION, FALSE);
nvOSResult = Err.Number;
This isn't a security issue as much as it is a sort-of scripting-call-Win32 issue. This scripting, not C++, so my suspicion is that InstallScript can't match to the types expected for the function - they may not exist in that world.
If they support it, then it's generally much easier to create a C++ Dll call - it can hardly be worse than this and you do get to include the header files that actually define those parameters correctly.