VC++ 2008 generating corrupted DLL?

Question

I've compiled a quite large project with VC++ 2008 (GNU Octave, http://www.octave.org), but I'm experiencing a crash in one of the DLL (liboctave), and when debugging it, it looks like the DLL is corrupted. For the record, all code is compiled against MSVCR80.DLL and with the flags: -EHsc -MD -Zi -fp:strict -fp:except-.

The test code is the following and the crash occurs at the "c = (a / b)" line (octave_in64 is typedef'ed to octave_int<__int64>)

#include <octave/config.h>
#include <octave/oct-inttypes.h>
#include <iostream>

int main (int argc, char **argv)
{
  double a (1.0);
  octave_int64 b (1), c;

  c = (a / b);
  std::cout << c << std::endl;

  return 0;
}

When debugging in VS2008, when reaching the offending line, I see the following in the disassembly window (the current code position is marked with >>>):

	  octave_int64 b (1), c;
	0040107B  mov         dword ptr [ebp-1Ch],1 
	00401082  lea         eax,[ebp-1Ch] 
	00401085  push        eax  
	00401086  lea         ecx,[b] 
	00401089  call        octave_int<__int64>::octave_int<__int64><int> (401032h) 
	0040108E  lea         ecx,[c] 
	00401091  call        octave_int<__int64>::octave_int<__int64> (401005h) 

	  c = (a / b);
	00401096  lea         ecx,[b] 
	00401099  push        ecx  
	0040109A  lea         edx,[a] 
	0040109D  push        edx  
	0040109E  lea         eax,[ebp-28h] 
	004010A1  push        eax  
>>>	004010A2  call        dword ptr [__imp_operator/<__int64> (406344h)] 
	004010A8  add         esp,0Ch 
	004010AB  push        eax  
	004010AC  lea         ecx,[c] 
	004010AF  call        octave_int<__int64>::operator= (401028h) 

When I step into (the code for operator/<__int64> is contained in the liboctave DLL), I see the following (current code position is again marked with >>>):

	mx_el_lt:
	1000100E  mov         esi,6700000h 
	do_ms_binary_op<octave_int<signed char>,octave_int<signed char>,float>:
	10001013  add         byte ptr [eax],al 
	Array<std::complex<float> >::checkelem:
	10001015  les         edi,fword ptr [eax+eax] 
	mx_inline_cummax<octave_int<int> >:
	1000101C  add         byte ptr [eax],al 
	operator/<__int64>:
>>>	1000101E  add         byte ptr [eax],al 
	operator/<__int64>:
	10001020  add         byte ptr [eax],al 
	operator/<__int64>:
	10001022  add         byte ptr [eax],al 
	DASRT_options::operator=:
	10001024  inc         eax  
	DASRT_options::operator=:
	10001025  add         byte ptr [eax],al 
	DASRT_options::operator=:
	10001027  inc         edx  
	mx_inline_and_not<octave_int<unsigned short>,octave_int<unsigned int> >:
	10001028  jmp         mx_inline_and_not<octave_int<unsigned short>,octave_int<unsigned int> > (106A2A30h) 
	operator==<__int64,unsigned __int64>:
	1000102D  jmp         operator==<__int64,unsigned __int64> (1065BF30h) 
	mx_inline_le<double,octave_int<unsigned __int64> >:
	10001032  jmp         mx_inline_le<double,octave_int<unsigned __int64> > (105E3290h) 

As you can see, the code it jumps to is not a JMP statement as I would expect. The code will execute until jumping to the completely unrelated mx_inline_and_not function and eventually crash.

I compiled the same code under VS2010 and the problem does not occur.

I am a bit puzzled about what's going on, as well as out of ideas. Could it be possible that VC++ 2008 generates invalid code? Any hint or suggestion about how to debug this further is welcome.

Answer 1

I've compiled a quite large project with VC++ 2008 (GNU Octave, http://www.octave.org), but I'm experiencing a crash in one of the DLL (liboctave), and when debugging it, it looks like the DLL is corrupted. For the record, all code is compiled against MSVCR80.DLL

..........

This is strange. For VC++ 2008, you want to use MSVC*90.DLLs. MSVC80* DLLs are for VC++ 2005.  VC2010 has compatibility mode with 2008 runtime, but not 2005.

-- pa

Answer 2

This is strange. For VC++ 2008, you want to use MSVC*90.DLLs. MSVC80* DLLs are for VC++ 2005.  VC2010 has compatibility mode with 2008 runtime, but not 2005.

-- pa

My mistake. Obviously, I meant msvcr90.dll. I just wanted to say everything is compiled with the -MD compilation flag. Just got confused by the 8 in 2008.

Answer 3

Hi goffioul,

According to your description, I'd like to suggest you move to GNU Octave Repository Forum for more helpful ideas at this link: http://sourceforge.net/projects/octave/forums. And this thread will be moved to Off-Topic Posts since we only discuss issues regarding the C++ language, compiler, and linker in Visual C++ Language forum.

Best regards,
Helen Zhao

---

Helen Zhao [MSFT]
MSDN Community Support | Feedback to us

Answer 4

Hi goffioul,

According to your description, I'd like to suggest you move to GNU Octave Repository Forum for more helpful ideas at this link: http://sourceforge.net/projects/octave/forums. And this thread will be moved to Odd-Topic Posts since we only discuss issues regarding the C++ language, compiler, and linker in Visual C++ Language forum.

I don't see how you can consider this off-topic or that the Octave source code could be at fault. It looks to me that the compiler generates invalid relocation in the octave DLL and makes client code to jump at invalid position in the DLL. For the record, this issue only occurs with the VC++ compiler, other compilers on Windows or other OS work fine.

If you look at the last disassembly snippet in my original post (which is a snapshot of the Visual Studio windows), you clearly see that the interleaved debugging info and assembly code do not match at all.

I have made some further testing and the issue does not show up when compiling with -O2 flag instead of -Zi. Moreover a stripped down conceptually similar example does not show the problem either.

MIchael.

Answer 5

It seems that binaries compiled with GNU toolchain are not very friendly to binaries produced by VC2008. There are certain technical reasons for this. One solution that I've seen in some opensource project documentation is to compile everyting with VC2008 (yes they have two binary versions, one compiled with VC2008 and another with gnu.).

VC2010 is different in this regard.  VC2008 will not be "fixed", because it is not broken.  If VC++2010 works for you, then everything is ok.

-- pa

Answer 6

It seems that binaries compiled with GNU toolchain are not very friendly to binaries produced by VC2008. There are certain technical reasons for this. One solution that I've seen in some opensource project documentation is to compile everyting with VC2008 (yes they have two binary versions, one compiled with VC2008 and another with gnu.).

VC2010 is different in this regard.  VC2008 will not be "fixed", because it is not broken.  If VC++2010 works for you, then everything is ok.

-- pa

I did compile everything with VC2008 (octave, all dependencies and client code). This is mandatory, because you can't mix different C runtime DLL. The fact that it works with VC2010 does not mean that the root cause has disappeared, I might just be lucky.

Michael.

Answer 7

If you recompiled everything with 2008, then it may be the /EHa vs /EHs syndrome. Need to check that the exception handling convention in the main program and DLL is same. (GCC is not aware of SEH at all? so use /EHs).

But... it just may be the progress: VC2010 is better than 2008 and supports more usages :)

-- pa

Answer 8

All C++ code is compiled with /EHsc flag.

Michael.

Answer 9

I still fail to see why this is considered off-topic. Please someone explain to me why symbol relocation is not the responsibility of the compiler/linker.

Michael.