Problem with dynamically generated .cs files

Question

I'm trying to use ANTLR but I'm having some problems with it's dynamically generated .cs files. ANTLR is a parser generator, used to work with domain-specific languages. You make a .g (grammar) file, which ANTLR compiles to four .cs files for use in your project. I've set the command that compiles the .g file to .cs files in the pre-build event of visual studio. So far so good. However, when I change the .g file and hit F5, the changes in the .g file are somehow not reflected in the code that gets run. I'm using a simple calculator example, and if I remove the required semicolon after a calculation, run the program and type 1+2 instead of 1+2; I still get an error complaining about the missing semicolon. I've tried deleting the generated files in the pre- and post-build events, I've tried deleting the bin and obj folders, but the only thing that seems to work is removing the generated .cs files from the solution and adding them again, which of course is a very annoying thing to have to do each time you change the grammar (probably about every 30 seconds when trying to build a new one).

Does anyone know a way to do this so I can just change the .g file, save, hit F5 in Visual Studio and have it use the most recent version of the generated files?

Answer 1

I don't think VS is going to detect a change in the .g file, so you'll probably need to do a rebuild rather than run (right click solution or project, rebuild).

Answer 2

Do you have the .g file added into the project? Have you set up how it is compiled?

 

Answer 3

The .g file is not part of the project, as naturally C# doesn't know what to do with it. The .g file is in the same folder as the .cs files. In the pre-build event, the command "java antlr.Tool -o ..\.. ..\..\calc.g" is run, which generates the four .cs files. I added these four files to the project, as they hold some class definitions without which the program won't compile. Just to be absolutely sure that these generated .cs files are up to date I'm deleting them in the post-build event. The project therefor consists of Program.cs and the four generated files, which naturally give an error when you try to open them, since the files aren't there. This doesn't matter as they're only required during building.

Suppose that the .g file says that a semicolon is required ("expr : mexpr (PLUS^ mexpr)* SEMI ;") when I open the project. Hitting F5 and typing 1+2 gives an error as expected. I then remove the SEMI word, save, go back to Visual Studio and hit F5. Typing 1+2 again produces an error, despite the fact that the .cs files have been regenerated. If I close and re-open Visual Studio or remove and re-add the four .cs files it does work correctly. Somehow it must be using a previous version of the code but I can't figure out how. Even explicitly building (F6 or project->Rebuild All) and running doesn't work either.

Can anyone tell me what I'm doing wrong?

Answer 4

It appears that things have changed in Visual Studio since the last time I used external build tools (VS 6). You used to be able to specify a compile tool and set dependencies on files in a project even when they were being compiled by an external tool.

 

I couldn't figure it out, but I did find this quote, "Pre-build events do not run if the project is up to date and no build is triggered."

 

I can't believe we would lose the ability to use external compile tools. There must be a way to set this up...

Answer 5

Ah.. I took one of my old projects and converted it to VS2008. Those custom build tools are still there, but it appears this feature is only available with C++ projects, not C#.

 

Answer 6

But surely a build would be triggered on F6 or Rebuild All?

In case it matters: I' using C# 2008 Express.

Answer 7

Unfortunately, you can add a .G file to your project but in order for Visual Studio to compile it you must have a Custom Tool written for it.  A custom tool as a managed assembly that contains a class that implements the IvsSingleFileGenerator interface.  You'd have to write something like that to get Visual Studio to compile it for you.

 

Alternatively, you can add something to the Pre-build event to "compile" certain *.g files.  See Project\properties\Build Events tab.

Answer 8

As I've mentioned, the command line to compile the .g file is already in the pre-build events. Compiling the .g file works, which can be verified by the fact that the generated .cs files reappear when running the program after having deleted them. The problem is that the compiled program doesn't seem to use the updated generated .cs files, as shown by the SEMI example given above.

Answer 9

Did some more testing. It works if I use the command line (first call java antlr etc, then csc). It also works in the IDE if I choose Build->Rebuild solution twice (just once doesn't work) before hitting F5. Surely there must be a way to be able to simply press F5 and have it work?

Answer 10

FalconNL2007 wrote:

Surely there must be a way to be able to simply press F5 and have it work?

 

Yes there is, Peter gave you the answer.

 

You don't have to build a compiler. Just an assembly that runs the compiler.

Answer 11

Apologies in case this issue has in fact been resolved,
but I don't understand the proposed solution...

I've got the exact same problem...

FalconNL2007 has described the situation, and maybe it can be
further clarified by taking a look at my build definitions:

<!-- the generated .cs files depend upon the .g file -->
<Compile Include="GrammarLexer.cs">
  <DependentUpon>Grammar.g</DependentUpon>
</Compile>
<Compile Include="GrammarParser.cs">
      <DependentUpon>Grammar.g</DependentUpon>
</Compile>

<!-- the .g file is part of the project and antlr will generate the two .cs-files from it -->
<Antlr3 Include="Grammar.g">
      <OutputFiles>GrammarLexer.cs;GrammarParser.cs</OutputFiles>
</Antlr3>

<!-- the antlr target will be executed before any build step -->
<BuildDependsOn>GenerateAntlrCode;$(BuildDependsOn)</BuildDependsOn>

<!-- the antlr target will execute the antlr tool exe (using the .g file as argument) -->
<Target Name="GenerateAntlrCode" Inputs="@(Antlr3)" Outputs="%(Antlr3.OutputFiles)">
   <Exec Command="AntlrTool-3.1.1.exe -message-format vs2005 @(Antlr3)" />
</Target>

Now what happens is:

1. edit the .g file (manually in Visual Studio 2008)
2. hit F6 (build)
3. the "GenerateAntlrCode" target is executed correctly, the two .cs files are correctly generated
4. the compiler is started correctly:
   "Csc.exe ... /out... GrammarLexer.cs GrammarParser.cs"

But, for some reason, in step 4 the compiler does not use the .cs files generated in step 3,
but their previous "version".
If I now touch the .g file and build again, the correct .cs files are used...

Can anyone explain why? Where does Studio store or cache the generated files?

And is implementing a wrapper for the antlr exe as a custom tool really the preferred solution?

Thanks a lot in advance!
Chris

 

Answer 12

Hi guys,

I managed to solve this problem with an awful bodge, but one that is quicker than writing a wrapper for your code generator.

Reasoning that I needed to generate cs files before VS got its hands on them, I put the call to the generator in the post-build event of a project in the same solution that I know will be built before the one that will contain the generated code.

My post-build event looks like this...

CodeGeneratingApp.exe parameter1 parameter2 > "$(SolutionDir)RealProject\GeneratedCode.cs"

If the real project is the first or only one to be built I create a dummy project just to access its post-build event, and make the real project dependant on the dummy one. I know this is horrible, but its a fast way of getting the desired functionality.

Answer 13

I looked at IVsSingleFileGenerator and it doesn't seem appropriate for this situation. Antlr produces two cs files from a single g file. IVsSingleFileGenerator claims a requirement for 1 output file for 1 input file.

I searched MSDN and can't seem to find any similar file generators. Will IVsSingleFileGenerator work for two files?