I have this problem with a project I am working on and I thought maybe asking a question here might give me some insight and possibly lead me to a solution.
I have a DLL that is written in C/C++ and it is required to assume __cdecl calling convention (MUST). I also wrote a wrapper DLL that calls into this dll with __stdcall calling convention. The reason I have this wrapper is because I have to use a VB6 application and based on suggestions from MS, this is the proper way to structure the code (http://support.microsoft.com/default.aspx/kb/153586?p=1)
I followed all the instructions but when I run the VB6 application from the IDE, it crashes with the Send/Do Not Send Error Report dialog box. When I comment the call from the wrapper dll to the main dll, the crash goes away.
Are there any project properties in VS2005 that I need to set in order for this to work without crashing?
Can anyone explain to me a little bit about how does the OS handle a DLL with a __stdcall calling convention calls a DLL with a __cdecl calling convention?
I am aware of and understand the difference between the two calling conventions but not sure what happens when they are mixed in a project like mine.
To be honest, if you followed the guide then there shouldn't be any problems. But the application crashing doesn't necessarily have to be because of calling convention issued. Have you wrote a sample C++ program to call the stdcall DLL to see if it works? You will also get the chance to debug it properly this way too.
As an example for you to test out.
__declspec(dllexport) int testfun()
__declspec(dllimport) int testfun();
__declspec(dllexport) int __stdcall testwrap()
__declspec(dllimport) int __stdcall testwrap();
int num = testwrap();
If you compile the first one into one dll, the second into a second dll linked with the first and then the third into an executable linked with the wrapper, then try it out it will work with no problems. Of course, I left all of the __declspecs inline to make things clearer and to keep things small.
Visit my (not very good) blog at http://c2kblog.blogspot.com/
Edited byCrescens2kSunday, April 04, 2010 11:11 PMAdded sample code
Marked as answer byNancy ShaoTuesday, April 13, 2010 5:40 AM