Is there a way to get MSBuild to use .sln files but determine dependency order for builds?

Question

I was thrilled to start playing with MSBuild to automate building our projects.  We have several solutions which build anywhere from 1-2 projects up to 50-60 projects.  One problem I've noticed however is that MSBuild doesn't seem to respect the dependencies to provide the proper build order when an MSBuild task loads a solution.  This sort of defeats the purpose of allowing the MSBuild task to load a solution if the build order is not respected.

Is there any way to tell MSBuild to respect the build order through project-to-project dependencies?  I poked around in the assemblies and it doesn't look like there is any way to handle this without a ton of work by the looks of it...

Answer 1

Can you give us a bit more detail? MSBuild will crack the solution file and will follow all the project-to-project references in the solution to ensure everything is built in an order that will allow all dependencies to get built first. What is a specific example of how your projects are hooked up and the order of the build isn't the way you expect?

Neil

Answer 2

I did notice that MSBuild would use the ProjectSection(ProjectDependencies) if they exist in the .sln file.  However, if the dependencies are not in the .sln, but rather only in the .csproj file, MSBuild just uses the .sln file's project order.

In particular, we have two new solutions created with VS 2005 (not imported from VS.NET 2003).  Both contain an application and a class library.  The application project uses the class library project as a dependency (referenced as a project).  Inside of VS 2005, the build order and dependencies are correct.  When the solution was created, the application project was added first and the class library project was added second and the application alphabetically comes first.  This means in the .sln file, the application is first and the class library second (either due to creation order or due to alphabetical order).  There is no ProjectSection(ProjectDependencies) in the .sln file.  The .csproj file for the application properly requires the class library project, so inside of VS 2005, the build order is correct based on dependencies.  When MSBuild loads the .sln however, the build order is wrong because of the order of the projects and the missing ProjectSection(ProjectDependencies) properties.

If I add the ProjectSection(ProjectDependencies) manually, it works as expected.

Is this a bug in the IDE then?  That the ProjectSection(ProjectDependencies) for out-of-order building is missing?  I can recreate this with a test .sln and projects.

Or is this a bug in MSBuild that it fails to load and interrogate the projects to ensure proper ordering based on dependencies?

Answer 3

Andy,
There are two kinds of ways to express project-to-project dependencies.
-- project to project references (Add Reference->Project tab) [this mechanism also handles gathering the referenced output file as well]. For .csproj, this is persisted as a <ProjectReference> item in the project file.
-- explicitly specified dependencies (Solution properties->Project Dependencies) [with this mechanism you would usually also add a File Reference to the referenced output yourself]. This writes the "projectdependencies" section in the .sln.

MSBuild on the command line will use both in order to figure the correct build order. If neither are present, its build order is undefined. Inside VS, if neither are present, the build order is also undefined and most likely different: it depends on implementation details inside VS and inside msbuild. We don't recommend you depend on the random project orderings VS and msbuild will give you in the absence of dependency information.

So it is possible to have a solution that builds okay in VS (possibly only in one particular version of VS) but not with msbuild because of missing dependency information. The solution to this is to correctly specify dependencies using either of those two mechanisms. You shouldn't need both, as you point out.

Having said all that, you mention that you do have a project to project reference and it still doesn't build in the correct order. That is odd. Can you check you have the appropriate <ProjectReference> items in your referencing .csproj's? If so maybe you can share your repro sln and projects -- zip and send to [e-mail] with a link to this thread please.

Thanks
Dan

Answer 4

Hi Dan,

  Im encounter the similar problem as the original user.   

I look at the files and I see the dependencies.   Im still confused on how to verify that the dependencies are set so that Team Build will be build the solution properly.

Can you clarify more or post examples?

 Email sent to email mentioned in the last post

Thanks,
Staffan

Answer 5

 

I have recently encountered a similar problem.  I have a solution with multiple projects (C#, C++, C++/CLI) using vs2005.  The dependencies use both project references and solution dependencies.  This would build fine in the IDE, but not on our build server using msbuild.  It was clear from examining the msbuilt log that it wasn't respecting the dependencies.  In the end I managed to resolve this problem by installing .net 2.0 SP1.

Answer 6

Hi Dan

We see the same problem. In our build we have worked around this by invoking devenv on the problematic .sln files, but not before spending a lot of time trying to make absolutely sure that our dependencies are correct in terms of reference type etc. I am tempted to claim that there is a bug in the dependency analysis of MSBuild (also in 3.5).

It's always the clean target that fail.

It seems to us that this behaviour might be related to projects that have COM registrations. For a long time I thought the problem was that the dependency was through COM, which of course MSBuild might have a hard time detecting. But now I see examples with a straight project dependency failing to build clean because it's done in the wrong order.

Martin Moe
Software Architect

Answer 7

We have the exact same issue.  Upgraded all on my solutions/projects to VS2008 and updated our build script to use msbuild 3.5 and it does not respect any project dependencies that are in place.  This impacts our batch build, continuous integration build and our 'official' build processes.  Anyone from Microsoft willing to admit to the defect and plans for resolution?

Answer 8

I just fixed this problem in my project, thanks AndyDragon, by adding ProjectSection(ProjectDependencies) section to sln file.
I blogged about this "MSBuild vs Visual Studio dependencies" issue in little more detail.

Answer 9

Hey I had this problem also, however, going through over 100 projects adding ProjectSection(ProjectDependencies) sections to each solution just was not practical for me, I never had to do this (since VS.NET 2002, none of these sections where ever in any of my .sln files). If anyone cares I did come up with my own work around and I figured I woudl post since the initial suggestion was hugely useful. I'm still working on find out what exactly is the issue, but this workaround did it for me....temporarily

Previously I only had Win32 code, thus when I built x86 code my command line was as follows:

msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Debug
msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Release

then I upgraded to .NET 3.5 and VS.NET 2008 sp1 still everything worked. Then I ported my code to x64 (not Ithium) because I wanted to run my Apps on MS Vista 64 bit...Figured no problem...

To do this it required some of my C++/CLI projects to differentiate the libs that were being included (32bit and x64). Therefore I need more than 1 platform so I had to specify. So I started doing the following at command line :

msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Debug;Platform=Win32
msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Debug;Platform=x64
msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Release;Platform=Win32
msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Release;Platform=x64

Well this had the unfortunate side effect as the problem that Andy Dragon described. Many of my projects in the solutions started building out of order all of a sudden..grrrrrrrrrr.

However I did notice that if I just went with the defaults and did not specify the platform everything worked (as before upgrading to .NET 3.5)

msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Debug
msbuild company.msbuild.xml /t:rebuildALL /p:Configuration=Release

So for me at least the issue was the for some reason Msbuild in .NET 3.5 CSharp projects do not like the /p:Platform=Win32 switch on the command line even though i specifically had Platform=Any CPU defined on the MSBuild element so I just modified my build script by adding this to the top like so:

<Project DefaultTargets="rebuildALL" ToolsVersion="3.5"
         xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<Choose>
    <When Condition=" '$(Configuration)'=='Release' ">
      <PropertyGroup>
        <Configuration>Release</Configuration>
      </PropertyGroup>
    </When>
    <Otherwise>
      <PropertyGroup>
        <Configuration>Debug</Configuration>
      </PropertyGroup>
    </Otherwise>
  </Choose>

  <Choose>
    <When Condition=" '$(Platform)'=='' ">
      <ItemGroup>
        <BUGFIX Include="Configuration=$(Configuration);Platform=Win32"/>
        <!-- Uncomment to build x64 <BUGFIX Include="Configuration=$(Configuration);Platform=x64"/> -->
      </ItemGroup>
    </When>
  </Choose>

....

</Project>

Then on my build targets I just substituted in for properties on the Msbuild element. For example:

<!-- for all my CSharp Files where x64/32 matters -->
 <Target Name="build_CPP">
    <MSBuild Projects = "..\..\somethingCPP.sln"
             Properties="@(BUGFIX)"></MSBuild>
  </Target>

<!-- for all my CSharp Files where x64/32 does not matter -->
 <Target Name="build_CSharp" DependsOnTargets="build_CPP">
    <MSBuild Projects = "..\..\somethingCSharp.sln"
             Properties="Configuration=$(Configuration);Platform=Any CPU"></MSBuild>
  </Target>

Note: the kludge here requires that BUGFIX contains all the key value pairs. For example you cannot do:  Properties="Configuration=$(ConfigurationName);SomethingProperty=someValue;@(CSharpMixed)".

Then when you need to switch between platforms you just uncomment out the x64 line and comment the Win32 line and everything works again (as shown above where the BUGFIX Item group is defined)

Again this was NOT a problem in VS.NET 2005. I only recently ran across it after upgrading to .NET 3.5 (and perhaps coincidentaly installed VS.NET 2008 sp1).

I have a very simple sample project that reproduces this if anyone wants it, just post on the board and I can upload.

Peace

Answer 10

I apologize for this bug. However, there should be a workaround that will be acceptable in some cases. You can change the <MSBuild> task to an <Exec Command="msbuild.exe ..." /> task, passing in the properties with the /p switch. This should make the solution get ordered correctly. This assumes that nowhere else in your build outside of the solution is concurrently doing other builds impacting projects in that solution -- if it is, then using msbuild.exe instead of <MSBuild> will get the two builds to trample on each other.

Dan

---

developer on msbuild

Answer 11

Guys, is it possible for MSBuild of the same version behave different on different PC?

I have a continious integration environment configured and it worked fine before I moved solution to MS VS 2008. I've changed MsBuild version to use, but the project compilation fails every time I force automatic build. But if I run MSBuild on my local PC the build completes successfully.

I looked into solution file and I can see that it miss ProjectSection(ProjectDependencies). Will adding these sections help, since it still compiles on local PC?

Answer 12

Is there any news on this?

I have the same annoying problem. MSBUILD 3.5 works perfectly against the solution file IF the machine has VS2008 SP1 installed but FAILS to deal with the build order if running on a machine with just the SDK 6.1 installed.

This is really infuriating and seems to have been a problem in MSBUILD since it was created. Any simple solution that doesn't entail installing VS2008 on my build server?

Ok, I'm in a bad mood after struggling with this and getting nowhere, but typical MS - offer something to replace a good open source tool, that seems to be just as good, but then have a really basic bug that is left unfixed for ages.

Please say there is a workaround....

Answer 13

AndyDragon  Thanks for discovering what causes the problem. I'm surprised you found this in 2005, it's now 2010 and this bug still seems to be living strong!

Answer 14

Hi Dan,

Is there a fix for this bug in the pipeline? We were considering rolling out and promoting MSBuild across our organization, but it's going to be a bit embarrassing doing that as we expect a number of teams would hit this problem.

The MSBuild version we're current trying is:

Microsoft (R) Build Engine Version 3.5.30729.1

[Microsoft .NET Framework, Version 2.0.50727.4200]

Regards,

   -Scott

Answer 15

I had this same problem, my issue turned out to be an invalid reference in one of my projects. It was a project I had used from another solution and one of the Project references was to a project that wasn't in the solution. It appears that MSBUILD gives up on determining the build order in this case.

Although this is probably by design, maybee in future versions there can be an option to disable this behavior. If VS can determine the correct order even with invalid references, MSBUILD should be able to.

Answer 16

I had same problem building a .sln file with TFS Build, and could reproduce it with a very simple solution: just one dll, one exe and one wix project for setup. The exe project was building before the dll one. I'd tried Project References, Project Dependencies and Project Build Order and everything was right, but the problem persist.

My workaround was very simple, so I share it here: I only changed order of projects on first lines of .sln file. Declaring dll before exe, it builds first on TFS Builds, and my problem was solved.

Pablo Núñez,
Málaga (SPAIN) 

---

Pablo Núñez, @pablonete

Answer 17

To be perfectly honest, it seems like this is another half baked, inconsistently implemented Microsoft solution that really isn't ready to be used universally in a build environment. I guess we're back to having to invoke devenv in our TFS build script (after having wasted hours on this problem).

Answer 18

Has Microsoft confirmed that this is a defect in MSBUILD 3.5?  I am seeing the same issue.  I have a solution with many projects.  Some projects have 'project references' to other projects in the solution.  In particular, the unit test wizard built into Visual Studio 2008 creates the unit test projects with a project reference to the project they are testing.  Builds in Visual Studio are fine.  Executing MSBUILD on the command line works on most machines but fails on a fair number of them.

I tried to create a simple example of a solution with 5 projects with a linear dependency chain.  The dependencies are all project references and do not match the order they were added to the solution (also the order listed in solution file) or the alphabetical order of project names.  The dependency chain also does not match the alphabetical order of the project GUIDs.  MSBUILD builds the solution without error, honoring the dependencies on every machine so far.