How to get custom build rules, ExcludedFromBuild and dependencies working together?

Question

I have a problem with using custom build rules, that so far every example I have found seems to suffer from the same issue.

Basically if you have a custom build rules setup for a file extension and have some of those files set to be excluded from build, I have yet to find an example that works correctly in this scenario. If I understand correctly what is happening is that the target's inputs contain the full list of input files, including those that have ExcludedFromBuild set to true so the dependency system thinks the target is dirty. Then there is a condition inside the command inside the target block that only executes the command if ExcludedFromBuild != true. Whilst the excluded files dont actually get built, it means it always tries to build them.

Interestingly it looks like visual studio itself has some kind of workaround for this, if I create a new VS project, add the provided masm build customisation, add 2 empty .asm files (just containing the word "END" so they assemble), mark one of them as dont build and then run msbuild from the command line it always spits out a message for assembling that file. If I instead try to build it from visual studio, visual studio itself somehow knows that the project is up-to-date so doesn't even call msbuild. (The fast dependency check I have heard mentioned).

Now I suspect that for some reason this fast dependency check isnt happening in my own case which is why I even started looking into this, but it seems like the core issue is that the provided masm aren't "correct", is there no way for msbuild to handle this scenario "correctly" ?

Answer 1

If we set ExcludedFromBuild as true, the excluded files will not be built.

We know Visual Studio uses the MSBuild project file format to store build information about managed projects. Project settings added and changed through Visual Studio are reflected in the .*proj file that is generated for each project. Visual Studio uses a hosted instance of MSBuild to build managed projects.

So I don't think there are some difference on building between using Visual Studio and Msbuild.

What you want to learn more you can refer to:

http://msdn.microsoft.com/en-us/library/dd393574

Answer 2

Andy,

MSBuild uses a timestamp to determine a project needs build. If input files' timestamp newer than output files' timestamp, it will trigger build. Otherwise it's up-to-date. As I understand, the custom target always trigger build when you build in commandline. You may post the script snippet here and get people analyze it.

thanks.

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 3

I'm not sure if this has largely become irrelevent now, as I have fixed my original issue although the behaviour still slightly bugs me. I had assumed that there was a way to query whether a complete msbuild job was up-to-date but it seems not. My original problem turned out to be down to visual studio's fast-up-to-date check where I was assuming even if that failed, if everything was correct and the project was up-to-date then msbuild should do nothing.

However, to explain my original complaint. If you create a new C++ project, I called mine testdep, add 2 .asm files both simply containing "END" and then drop to the command line and build twice you should get the following output

C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep>msbuild /detailedsummary
Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.269]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

Build started 15/08/2012 08:08:04.
Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Win32".
Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" (1) is building "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep\testdep.
vcxproj" (2) on node 1 (default targets).
InitializeBuildStatus:
  Creating "Debug\testdep.unsuccessfulbuild" because "AlwaysCreate" was specified.
_MASM:
Skipping target "_MASM" because all output files are up-to-date with respect to the input files.
_MASM:
Skipping target "_MASM" because all output files are up-to-date with respect to the input files.
ClCompile:
  All outputs are up-to-date.
  All outputs are up-to-date.
ResourceCompile:
  All outputs are up-to-date.
ManifestResourceCompile:
  All outputs are up-to-date.
Link:
  All outputs are up-to-date.
Manifest:
  All outputs are up-to-date.
FinalizeBuildStatus:
  Deleting file "Debug\testdep.unsuccessfulbuild".
  Touching "Debug\testdep.lastbuildstate".
Done Building Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep\testdep.vcxproj" (default targets).

Done Building Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" (default targets).

If you now, clean and then mark one of the asm files as excluded from build and then build twice the second output is as following

C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep>msbuild /detailedsummary

Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.269]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

Build started 15/08/2012 08:09:36.
Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Win32".
Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" (1) is building "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep\testdep.vcxproj" (2) on node 1 (default targets).
InitializeBuildStatus:
  Creating "Debug\testdep.unsuccessfulbuild" because "AlwaysCreate" was specified.
_MASM:
Skipping target "_MASM" because all output files are up-to-date with respect to the input files.
_MASM:
  Assembling [Inputs]...
ClCompile:
  All outputs are up-to-date.
  All outputs are up-to-date.
ResourceCompile:
  All outputs are up-to-date.
ManifestResourceCompile:
  All outputs are up-to-date.
Link:
  All outputs are up-to-date.
Manifest:
  All outputs are up-to-date.
FinalizeBuildStatus:
  Deleting file "Debug\testdep.unsuccessfulbuild".
  Touching "Debug\testdep.lastbuildstate".
Done Building Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep\testdep.vcxproj" (default targets).

Done Building Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" (default targets).

The key difference is it now always thinks one of the _MASM targets needs doing, its just when it executes that  it doesnt do anything (well other print a message).

No the masm file doesnt actually get assembled, and the assemble message can be removed if nescessary but fundementally the build system doesn't think everything is up-to-date.

My issue stemmed from assuming that that msbuild decision was used to decide whether visual studio thought a project was up-to-date or not. I have now clarified how that works and fixed my original issue so this isnt actually an issue any more, it just seems "icky" that a build system doesn't actually know if its up-to-date or not.

edit: Actually one thing this does affect is target batching, although i'm not using that at the moment, I think you would have to distingish batches based on exclude status.

Answer 4

I'm not sure if this has largely become irrelevent now, as I have fixed my original issue although the behaviour still slightly bugs me. I had assumed that there was a way to query whether a complete msbuild job was up-to-date but it seems not. My original problem turned out to be down to visual studio's fast-up-to-date check where I was assuming even if that failed, if everything was correct and the project was up-to-date then msbuild should do nothing.

However, to explain my original complaint. If you create a new C++ project, I called mine testdep, add 2 .asm files both simply containing "END" and then drop to the command line and build twice you should get the following output

C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep>msbuild /detailedsummary
Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.269]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

***

If you now, clean and then mark one of the asm files as excluded from build and then build twice the second output is as following

C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep>msbuild /detailedsummary

Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.269]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

Build started 15/08/2012 08:09:36.
Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Win32".
Project "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep.sln" (1) is building "C:\Users\andy\Documents\Visual Studio 2010\Projects\testdep\testdep\testdep.vcxproj" (2) on node 1 (default targets).
InitializeBuildStatus:
  Creating "Debug\testdep.unsuccessfulbuild" because "AlwaysCreate" was specified.
_MASM:
Skipping target "_MASM" because all output files are up-to-date with respect to the input files.
_MASM:
  Assembling [Inputs]...
ClCompile:
  All outputs are up-to-date.
  All outputs are up-to-date.

***

Hi,

I'll try to get other people help explain the behavior. btw, I don't find how to add asm file to C++ project yet in a fast try, if you could share that would be great.

thanks,

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 5

Simply add new file as normal, select .cpp or .h as the filetype but then change the extension to .asm, it should add the file with a .asm extension. Either that or you can just create the 2 asm files in explorer and add existing item to the project to get them in there.

edit the two files and just put on a single line the word "END" (minimum needed to keep masm happy)

Right click the project in solution explorer, choose build customizations and select masm(.targets, .props)

right click each of the .asm files you added to the solution, select properties and change item type to Microsoft Macro Assembler

now hit build! it should assemble both files and compile and link everything fine.

Answer 6

OK. I try to get other people to help this query, please wait some time.

thanks.

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 7

This kind of request needs research and trial and error mechanism and I'm afraid that from a support perspective this is really beyond what we can do here in the forums. If you cannot determine your answer here or on your own, consider opening an ADVISORY support case with us. Visit this link to see the various support options that are available to better meet your needs:  http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophone.

---

Trevor Hancock (Microsoft)
Please remember to "Mark As Answer" the replies that help.

Answer 8

Wow, what?

So first there is a complete lack of any form of UI for actually editing build customisations in VS 2010 forcing me to actually do this stuff manually compared to 2008.

Secondly there is the lack of any decent documentation, instead most of what I have learned so far has been from the msbuild book since the actual official docs are so bad and from manual experimentation.

So when I ask how to actually get a custom build rule working properly with items that have been marked "excluded from build" (A presumably "official" flag in the IDE) rather than the hack that's used in the (only) provided MASM example, I get told the answer is only available via paid for support?

Pretty poor show... Its not like I'm asking anything about my own stuff which I would expect to have to get 1-2-1 support for, this entire thread based on the provided MASM build rule.

If that's all the information I'm going to get then fair enough, but I'm distinctly unimpressed that something that just works in 2008 is apparently unsupported out of the box and requires paid for support to find out.

Answer 9

Okay Im still having trouble with this, let me see if I can ask it in a way that doesn't involve experimentation and hope I can get a reasonable answer.

How do I combine batching and a task conditional based on item metadata?

The dependency check is done on the target level, and then there are conditions on the tasks in that target to not run the task based on a condition.

The problem is if I naively batch things together then the dependency check on the target batches everything together and decides the excluded things are dirty (as its destination doesnt exist). I can batch on the task so that we don't actually run the target on the items whose condition is false but it still runs the task on those for which the condition is true, since it has already run the dependency check.

Basically I cant find anywhere in the docs that tells me how to get batching at the target level on arbitrary item metadata so that I get correct dependency checking.