none
vcbuild silently truncates multithreaded build

    Question

  • When building a VS2008 solution of C++ projects (non-managed) using "vcbuild /M4", the last couple of projects being built seem to be silently truncated and not fully built.

     

    I noticed this on a build server which is using /M4. It's a dual cpu, dual core machine running Windows Server 2003. The build script copies all .dll files from the output folders into a deployment dir and we recently noticed that some dllTongue Tied were missing from the builds. We found traces of them, like .ilk files and some of the object files, as well as *.rsp files in the intermediate folder. The *.rsp files weren't complete, they were missing half of the content and should usually have been deleted during the build process. Especially the link step was missing.

     

    After some experiments we noticed that we could replicate this behaviour on most machines when using /M4, regardless of the number of cores available. The log was also incomplete, this is how the end of the build log looked on my dual core machine. As you can se there is no output indicating that projects Foo and Bar have been completed and only the stdafx.cpp files are being compiled.

     

    27>Build log was saved at "file://D:\ws\betstone-main-axl\Local\Build\Ferret\Debug\Client\BuildLog.htm"
    27>Client - 0 error(s), 0 warning(s)
    55>Build started: Project: Foo, Configuration: Debug|Win32
    55>Compiling...
    55>stdafx.cpp
    49>Build log was saved at "file://D:\ws\betstone-main-axl\Local\Build\Ferret\Debug\Games\ExternalModule\BuildLog.htm"
    49>ExternalModule - 0 error(s), 0 warning(s)
    57>Build started: Project: Bar, Configuration: Debug|Win32
    57>Compiling...
    57>stdafx.cpp
    51>Build log was saved at "file://D:\ws\betstone-main-axl\Local\Build\Ferret\Debug\Test\Plugins\Project5.Test\BuildLog.htm"
    51>Project5 - 0 error(s), 0 warning(s)
    Build complete: 26 Projects succeeded, 0 Projects failed, 0 Projects skipped

     

    So, it looks like when the first "thread" is finished building its part of the project, it silently terminates the remaining threads that may still have things left to build.

     

    Running the build again with /M4 doesn't fix the missing files. Running again with /M2 however, builds both the missing projects.

     

    As far as I can tell this worked well in VS 2005 and we can trace the missing files back to when we upgraded the project to VS 2008.

     

    vcbuild version:

    Microsoft (R) Visual C++ Project Builder - Command Line Version 9.00.21022

    Wednesday, May 14, 2008 5:01 PM

Answers

  • Ugh, that's fugly.  There are numerous problems with concurrent builds in VS2008, centered around mspdbsrv.exe and the /MP compiler option.  This is a new one I haven't heard of before.  If you can wait a couple of months, try the VS2008 SP1 service pack.  It is in beta right now if you want an early peek.  Post to the Connect web site if you can wait a week or so.  Call Microsoft CSS if it is urgent.
    Thursday, May 15, 2008 11:24 AM

All replies

  • Ugh, that's fugly.  There are numerous problems with concurrent builds in VS2008, centered around mspdbsrv.exe and the /MP compiler option.  This is a new one I haven't heard of before.  If you can wait a couple of months, try the VS2008 SP1 service pack.  It is in beta right now if you want an early peek.  Post to the Connect web site if you can wait a week or so.  Call Microsoft CSS if it is urgent.
    Thursday, May 15, 2008 11:24 AM
  •  

    Thanks, I've turned off the /M option for now and I'll post the problem to Connect.

     

    /axl

    Thursday, May 15, 2008 12:39 PM
  • Ugh ! We are getting this also in VS2005-SP1 (we can not yet move to VS2008 so have not tried it there). We have seen it on both XP-SP2-x64 (quad-core) and Win2k3-R2-x64 (dual quad core), and with /M8 and /M4. The solution has ~45 VCPROJ files, about 7 of which are excluded from the build. It would seem that the same projects fail to start or silently end each time, even when all are "up-to-date".

    We have many solutions, and only one is exhibiting this, currently. Note we do have one where the inter-project dependencies are not truly honoured, which leads to other build errors. We really need to reduce our build times, so using /M8 is pretty important to us.

    So far I'm unable to create a simple solution which reproduces the problem, but if I can I'll raise a call and submit.

    I'm desparate for some ideas to move forward as we're in the process of moving about 8000 source files from a "make" based build to a VCPROJ file & solution file based build. Contact welcome at +1.604.453.4330 or mailto:rob[dot]dalsanto[at]kodak[dot]com

    While I'm at it, we've also noted that /M3, /M5, /M6, /M7, /M9, ... i.e. anything but a power of 2 will cause VCBuild to use "/M1" (we can tell because the output is not preceded by the "[projectNum]>").

    BTW, I can see Microsoft intends not to fix this but to let us suffer until VS2010 is out, at least as of Nov 18/08 at https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=344257 Arghhh !
    Friday, December 05, 2008 8:13 PM