none
Visual Studio Caches Source Files before build starts

    Question

  • Really, this question belongs in a Visual Studio Build System forum, but there is none, so the Debugger is the next closest thing because the effects can be seen in the debugger.

    Here is the problem.  When you start a build in visual studio, visual studio will cache the contents of all compilable source files (those with a Compile msbuild node).  If any of your source files get modified during the build, then these source modifications will not be used during the build.  The modifications will show up in the editor, but not in the assemblies.  The result is that when debugging, your source will differ from what was used to generate the debug assemblies and the debugger will complain about this.

    Before anyone asks, a good example of when this would happen is when you are generating source files from a template (or in our case a grammar spec).  I have read elsewhere that this problem exists for T4 templates that generate source files, but i have not verified this.

    Finally, it doesn't matter if you perform the source generation steps earlier in the build (i.e. before the compile) because the caching occurs at the very instant that you start a build.  By this i mean that even if you inject your msbuild target as a dependency of the BeforeBuild target, it won't matter, because even in the before build target the source files have already been cached and any updates to them will not be used in this build.

    The current solution is to simply build twice in a row, but this is by all means a terrible solution.  If anyone knows of a way to force visual studio to ignore cached source files, or force a refresh of source files, etc... this would be a great help.

    Finally, this is not related to the msbuild system, as this problem does not exist when invoking msbuild from the command line.  It is purely visual studio related, which makes me wonder if there is some configuration option somewhere that can be disabled so that source file caching is not used.
    Wednesday, November 18, 2009 9:07 PM

Answers

  • I found the answer.  The issue is that there are two compilers that you can use in visual studio, an internal compiler and the external compiler (the one msbuild uses).  The internal one, as you can imagine, caches input files so that it is more efficient.

    To force visual studio to use the external compiler, you need to add the following to your msbuild file:

    <PropertyGroup>
      <!-- Force visual studio to use external compiler and avoid cached files -->
      <UseHostCompilerIfAvailable>False</UseHostCompilerIfAvailable>
    </PropertyGroup>

    From what i have read, you should add this property immediately above your csharp targets import line, looks like this:

    <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />

    After adding the property to your msbuild file, visual studio will use the external compiler when generating csharp assemblies and will use the source as it exists at the point of compilation, rather than at the point of build invocation.

    Note that you should not do this on a large project as your compile times will increase quite a bit (not verified, but heavily speculated).  Instead, you should extract any functionality that is automatically generated (plus any dependent source files) into a separate project to minimize the impact of the external compiler.  As far as i can tell, this property cannot be modified during a build, so you either choose to use the external compiler for all compilations or no compilations.

    EDIT: i should give credit to the source for this answer, found it on Sam's Blog
    • Marked as answer by Pat Sissons Thursday, November 19, 2009 6:23 PM
    • Edited by Pat Sissons Thursday, November 19, 2009 6:24 PM linking url where i found the answer
    Thursday, November 19, 2009 6:23 PM

All replies

  • Hello Pat,

    Thank you very much for posting threads about Visual Studio in our forum.

    I'm not sure of the version of your Visual Studio. But if it is Visual Studio Team System 2008, I would recommend you go to Tools -> Options -> Projects and Solutions -> Build and Run and set Before building box to Save all changes option.

    An alternative way is to reset the settings of Visual Studio, please go to Visual Studio 2008 command prompt and run devenv.exe /resetsettings.

    Please have a try and let me know the result,
    Thanks a lot
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    If you have any feedback, please tell us.
    Welcome to the All-In-One Code Framework!
    Thursday, November 19, 2009 8:21 AM
  • it is indeed VS.NET TS2008, so i will give this a shot.

    EDIT:
    I don't think this will actually help, as this will just save any unsaved changes before the build starts, which is too early.  (btw, the option was set to Save all changes anyways).

    here is the order of operations, as far as i can tell:

    [build event executed] --> [all *.cs files cached] --> [Generate new asdf.cs file (dependency of BeforeBuild target)] --> [Compile *.cs files (uses cached files)] --> [Output to assembly]

    we need to force the Compile step to refresh its cached *.cs files, so that our generated asdf.cs file is used in the compile.  If asdf.cs does not exist when the build event occurs, then there is no problem because there is no file to cache and the generated file must be used.  If asdf.cs DOES exist, on the other hand, we see the behavior described above.
    Thursday, November 19, 2009 5:18 PM
  • I found the answer.  The issue is that there are two compilers that you can use in visual studio, an internal compiler and the external compiler (the one msbuild uses).  The internal one, as you can imagine, caches input files so that it is more efficient.

    To force visual studio to use the external compiler, you need to add the following to your msbuild file:

    <PropertyGroup>
      <!-- Force visual studio to use external compiler and avoid cached files -->
      <UseHostCompilerIfAvailable>False</UseHostCompilerIfAvailable>
    </PropertyGroup>

    From what i have read, you should add this property immediately above your csharp targets import line, looks like this:

    <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />

    After adding the property to your msbuild file, visual studio will use the external compiler when generating csharp assemblies and will use the source as it exists at the point of compilation, rather than at the point of build invocation.

    Note that you should not do this on a large project as your compile times will increase quite a bit (not verified, but heavily speculated).  Instead, you should extract any functionality that is automatically generated (plus any dependent source files) into a separate project to minimize the impact of the external compiler.  As far as i can tell, this property cannot be modified during a build, so you either choose to use the external compiler for all compilations or no compilations.

    EDIT: i should give credit to the source for this answer, found it on Sam's Blog
    • Marked as answer by Pat Sissons Thursday, November 19, 2009 6:23 PM
    • Edited by Pat Sissons Thursday, November 19, 2009 6:24 PM linking url where i found the answer
    Thursday, November 19, 2009 6:23 PM