none
Generated Code Not Building - VS Caching Problem? RRS feed

  • Question

  • Hi,

    I have a custom MSBuild task that generates code. The code is being generated OK, but I'm having trouble getting VS integration working properly. When the code is generated, it is seemingly not being fed into the compiler. I have to do a rebuild or modify the generated file myself to get VS to build it.

    I'm at a loss as to why VS isn't recognizing the generated file as changed and requiring a build. The diagnostic output for MSBuild even shows that csc.exe is being invoked correctly, and the generated file is being passed in. However, it's as if csc.exe is seeing the version of the file before generation. If I instead do the build myself at the command line (copying the exact command from the MSBuild output), everything works as expected.

    Perhaps VS is caching a copy of the file in memory? Is there anything obvious I have to do after generating the file to ensure it is "refreshed" so VS knows it has been changed?

    Thanks,
    Kent
    Thursday, January 24, 2008 8:52 PM

Answers

  • YES!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Sorry, I'm excited coz I just figured it out.

    Building from the command line worked fine. I could make a change to the template, run msbuild again and - bingo - the change was present in the EXE. That proved it was some kind of caching in VS.

    I tried specifying UseHostCompilerIfAvailable=false to ensure the compilation happended outside of the VS process and that works! Of course, I it's less efficient, but the thing is I understand it applies only to that project and the generated code rarely changes.

    If anyone has any further input please let me know.

    Thanks,
    Kent
    Thursday, January 24, 2008 9:21 PM

All replies

  • YES!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Sorry, I'm excited coz I just figured it out.

    Building from the command line worked fine. I could make a change to the template, run msbuild again and - bingo - the change was present in the EXE. That proved it was some kind of caching in VS.

    I tried specifying UseHostCompilerIfAvailable=false to ensure the compilation happended outside of the VS process and that works! Of course, I it's less efficient, but the thing is I understand it applies only to that project and the generated code rarely changes.

    If anyone has any further input please let me know.

    Thanks,
    Kent
    Thursday, January 24, 2008 9:21 PM
  • I ran into a similar issue the other day. As far as I can tell, when changes are made to a source code file on-disk during the build process, Visual Studio doesn't seem to pick up the changes until the next build.

    In other words, if I manually modify something in the generated source code file to introduce an error, and then attempt to build, that error will be reported even though the file is regenerated before the compiler was actually invoked. However, if I then build again, it will see the regenerated content and build successfully. (My generator doesn't change the generated source code file at all during this second build, since it is already up-to-date.)

     

    There are two workarounds to this that I have been able to find:

    1) As you mentioned, setting the UseHostCompilerIfAvailable property to false causes MSBuild to not use the in-process compiler, which bypasses the issue. Unfortunately, as you also noted, not using the in-process compiler hurts build performance.

    2) The IVsMSBuildTaskFileManager host object can be used to directly modify VS's in-memory cache of the generated source code file, allowing it to "see" the changes immediately during the build process. I can post some code that shows you how to do this if you'd like.

     

    The drawback to the second approach is that (as far as I have been able to determine) your Target and Task need to be registered with each project type (e.g. C#, VB, etc.) that will need to use them in order to tell the project system to pass in an instance of that interface as the Task's HostObject. In my case, this really complicates installation, since now my installer not only needs to find every installed variation of VS to register the targets file as a "SafeImport," it also needs to find every project type with which my extensions may be used in order to register the appropriate Target/Task pairs. Plus, anyone who wants to use the Task in another Target will need to go and register that pair themselves.

    (To see where this registration is performed, search for "MSBuildHostObjects" in the VS registry under the "Projects" registry key. The format of the subkey names seems to be "TargetName;TaskName".)

    Friday, January 25, 2008 6:31 PM
  • Awesome - thanks for the info Kevin. Based on that, I think I'll stick with the out-of-process compiler.

    I have a single project that is using code generation. It will rarely change, and so in most cases the compiler won't even be invoked. It's good to know there's a workaround, though, as I may need it in the future.

    Thanks again,
    Kent
    Friday, January 25, 2008 9:15 PM
  • Thanks too, Kevin - at least that is a fix!I just ran into the same issue using a pre build step in vs to update assy versiond (UpdateVersion).
    I wonder - as this post is a year old - if anything has been happend on that frontline in the meantime?
    The option 1 above works - but it seems it have side effects on intellisense - at least it seems I get lots of additional output and complaints.
    I wonder why there just isn't a recheck on system file status invoked by MSBUILD but - being a relative occasional MSBUILD user I may not see what the issue is to admit - but what sense does a pre build step make if a simple source mod causes such hickups....
    Anyways: any hint is appreciated!
    Thanks
    tb
    Friday, March 13, 2009 11:44 PM
  • I found an interesting workaround :) Computer are stupid, infact, they do what humans ask.
    This workaround is so simple in concept that when I had the idea I was impressed it worked!
    I would like to know if this works for you.

    The solution is based on a custom task that add some spaces in file names to confuse the visual studio hosted compiler cache :)
    I keep the number of spaces to use in a static dictionary (tasks dll are not unloaded by visual studio) so I provide always a different number of spaces from previous compilation.
    This ensure that visual studio cache is not used on newly generated files.
    If you close and reopen visual studio, this will works too :)
    This works also on visual studio 2008 express, that don't allow the use of visual studio SDK.
    I provided static functions so you can use the same functionality in your custom tasks.
    The code can compile both in 2005 and 2008.

    What do you want more? :)

    Please, let me know if it works for you!

    using System;
    using System.Collections.Generic;
    using Microsoft.Build.Utilities;
    using Microsoft.Build.Framework;
    
    namespace SP.MSBuild.Tasks
    {
        /* Sample usage
       
        <UsingTask TaskName="VSFileCacheWorkaround" AssemblyFile="C:\MyPath\SP.MSBuild.dll" />
    
        <Target Name="MyTarget" Inputs="@(MyInputs)" Outputs="@(MyInputs -> %(RootDir)%(Directory)%(Filename).cs)">
          <MyTransformationTask Input="%(MyInputs.FullPath)" Output="%(MyInputs.RootDir)%(MyInputs.Directory)%(MyInputs.Filename).cs" ContinueOnError='true' />
          <VSFileCacheWorkaround Prefix="MyTarget" Input="%(MyInputs.RootDir)%(MyInputs.Directory)%(MyInputs.Filename).cs">
            <Output TaskParameter="Output" ItemName="Compile" />
          </VSFileCacheWorkaround>
        </Target>
       
        */
    
        /// <summary>
        /// Author: Salvatore Previti - 08-April-2009.
        /// 
        /// Visual studio host compiler keeps a cache of files during the MSBuild step, so they are not readed from disk.
        /// This is ok for performance and intellisense, right, but this introduces also some subtle bug!
        /// 
        /// See forum thread "Generated Code Not Building - VS Caching Problem?" at http://social.technet.microsoft.com/Forums/en-US/msbuild/thread/14bb304d-d8e6-4227-bc2b-e2e6d4f979be?ppud=4 for more informations.
        /// This task contains a simple but effective work around to this caching bug :)
        /// 
        /// How it works:
        /// 
        /// Visual studio keeps MSBuild Tasks dll loaded also between different compilations in the same appdomain where MSBuild is executed.
        /// Due to this fact, we can keep some states (static variables) between different compilations during the same development session.
        /// This class, infact, uses a static synclocked dictionary to keep states between different compilations.
        /// Each time static function UpdateFileUsageCount is called with the same prefix and with the same filename (case insensitive), the resulting value is incremented.
        /// 
        /// There is another "feature" (bug?) in visual studio cache mechanism... &lt;Compile Include="c:\test.cs"&gt and &lt;Compile Include="c:\test.cs "&gt
        /// are considered different files, so, they are cached as two different files (look the space character in "test.cs " in the second Compile tag).
        /// MSBuild and all filesystems, instead, treat both files as the same :)
        /// 
        /// So the solution! Each time we generate a file, we use a different number of spaces using our in-ram dictionary.
        /// The static function that does this is UpdateSpaces.
        /// During tests, a single space seems sufficient, but to be safe, I used MaxSpaces constant to use more than one space.
        /// 
        /// This system has some drawback, though:
        ///     1) You need to use this Task, so, you have to include a DLL and add some tags to your MSBuild project.
        ///     2) This should works for full pathed filenames.
        ///     3) Not garbage collector friendly - some objects (internal static dictionary and some Visual Studio cached files) will never be released and there is no way to release them.
        ///        This is a small amount of RAM, though, and should not be a problem during development.
        ///        During command-line MSBuild this problem can be ignored.
        /// </summary>
        public class VSFileCacheWorkaround :
            Task
        {
            #region Constants
    
            /// <summary>Max number of spaces to use in <see cref="UpdateSpaces"/></summary>
            public const int MaxSpaces = 5;
    
            #endregion
    
            #region Fields
    
            private string prefix;
            private string inputFileName;
            private string outputFileName;
            private bool enabled;
    
            #endregion
    
            #region Constructors
    
            public VSFileCacheWorkaround()
            {
                this.inputFileName = string.Empty;
                this.outputFileName = string.Empty;
                this.enabled = true;
            }
    
            #endregion
    
            #region Properties
    
            /// <summary>
            /// Enable or disable work around.
            /// If disabled, function UpdateSpaces() will be not used in Execute() and 0 spaces will be added.
            /// </summary>
            public bool Enabled
            {
                get { return this.enabled; }
                set { this.enabled = value; }
            }
    
            /// <summary>The prefix to use to differentiate cache access. Default value is null.</summary>
            public string Prefix
            {
                get { return this.prefix; }
                set { this.prefix = value; }
            }
    
            /// <summary>The input filename.</summary>
            [Required]
            public string Input
            {
                get { return this.inputFileName; }
                set { this.inputFileName = value; }
            }
    
            /// <summary>Output filename. 0 or more spaces will be added to <see cref="FileName"/></summary>
            [Output]
            public string Output
            {
                get { return this.outputFileName; }
            }
    
            #endregion
    
            #region Overrides
    
            /// <summary>Execute the task.</summary>
            public override bool Execute()
            {
                this.outputFileName = this.enabled ? UpdateSpaces(this.prefix, this.inputFileName) : this.inputFileName;
                return true;
            }
    
            #endregion
    
            #region Static
    
            private static Dictionary<string, int> fileUsageCountTable = new Dictionary<string, int>(StringComparer.InvariantCultureIgnoreCase);
    
            /// <summary>
            /// Given the same prefix and a filename, result value is always different (incremented) each time this function is called.
            /// A static synclocked dictionary is used.
            /// String comparison is case insensitive.
            /// Used by <see cref="UpdateSpaces"/>
            /// </summary>
            /// <param name="prefix">A prefix to use, or null</param>
            /// <param name="filename">The file name</param>
            /// <returns>An integer number equals to or greater than 0 that is incremented each time this function is called with the same parameters (case insensitive)</returns>
            public static int UpdateFileUsageCount(string prefix, string filename)
            {
                int result = 0;
                if (!string.IsNullOrEmpty(filename))
                {
                    filename = filename.TrimEnd();
                    if (filename.Length > 0)
                    {
                        if (!string.IsNullOrEmpty(prefix))
                            filename = prefix + '|' + filename;
                        lock (fileUsageCountTable)
                        {
                            fileUsageCountTable.TryGetValue(filename, out result);
                            fileUsageCountTable[filename] = result = unchecked((result + 1) & 0x7FFFFFFF); // Increment but keep positive.
                        }
                    }
                }
    
                return result;
            }
    
            /// <summary>
            /// Add some spaces after the specified <paramref name="filename"/>.
            /// Number of spaces is calculated using <see cref="UpdateFileUsageCount"/> so
            /// number of spaces changes between two (or some more) different calls with the same parameters (case insensitive).
            /// </summary>
            /// <param name="prefix">A prefix to use, or null</param>
            /// <param name="filename">The file name</param>
            /// <returns><paramref name="filename"/> + 0 or more spaces</returns>
            public static string UpdateSpaces(string prefix, string filename)
            {
                int spaces = UpdateFileUsageCount(prefix, filename) % MaxSpaces;
                return spaces > 0 ? filename + new string(' ', spaces) : filename;
            }
    
            #endregion
        }
    }
    

    Wednesday, April 8, 2009 3:48 PM
  • Salvatore Previti, thanks a lot for that idea!
    Wednesday, October 17, 2012 5:55 PM