locked
Pinned Fields in the Common Language Runtime

    Question

  • Hello,

    I'm attempting to write a compiler framework in C# that alters the perceived structure of the types to include high level constructs for the various kinds of types (enums, interfaces, et al.)  Into that, because there's a difference in structure between the framework I'm trying to write and the CLI itself, and the fact that Reflection.Emit has a few drawbacks, I'm writing a metadata parser (along with a metadata parser generator, to update the table references as I understand them better).  In writing your own metadata parser, you learn a few things about the CLI framework that you didn't know previously, such as function pointers that represent a type (mind you, not delegates), and one such anomaly is pinned fields.

    I haven't gotten to the method bodies, but one thing I'm confused by is stand alone signatures which start as a field signature (0x06) followed immediately by Pinned (ELEMENT_TYPE_PINNED - 0x45) which is followed by another type usually with the IsExplicitlyDereferenced C language optional type modifier.  Am I to assume that these field references represent memory locations that are pinned because they exist as a means to breach managed<->native barrier, and the information contained within this explicitly referenced but implicitly defined field as it is defined on the native C side?

    Any insight on this is appreciated.

    -Allen Copeland Jr

    Sunday, July 08, 2012 3:30 PM

Answers

  • This is an interesting related thread you may want to take a look at: http://www.netframeworkdev.com/building-development-diagnostic-tools-for-net/field-signatures-in-standalonesig-table-30658.shtml

    Karel and Rick on that thread are both developers on the CLR team, so you can trust those responses. The short answer sounded like officially that metadata is invalid according to spec, however something is (erroneously) generating it anyways. If you are particularly curious I can see if Karel ever found an answer.

    Tuesday, July 10, 2012 9:16 AM
    Moderator
  • Karel,

    Thanks for your insight.  I kind of figured that due to the link between the locals and the debugging symbols generated that the link went pretty deep.  My guess is the dependency has been in there a lot longer than anyone had anticipated.  The C++/CLI compiler emits the information whether the library is marked as debug or release, though the number of emitted entries changes, my guess is there's a much tighter dependency on the table than anticipated, the same goes for the C# compiler.  Hopefully as I research this further I might be able to figure out how to cause the C++/CLI to emit the pinned fields.  I only assume they're pinned because the signature contains the proper prolog for pinning, the signature's doesn't parse otherwise.  If you're interested I could give a signature index relative to a specific version of a framework library that emits a 'pinned field' signature within the table.

    Thanks,

    -Allen Copeland Jr.

    PS: The irony of finding C# emits field signatures in the StandAloneSig table is the clue involved the constant type-names I used in parsing the various signatures within the metadata.  They appeared sparsely throughout the code, and made it easier to track down.
    Tuesday, July 17, 2012 9:35 AM

All replies

  • Hi Alexander,

    I cannot understand you what do you want to do, please describe it clearly:

    What do you want to do? make a compiler like C#,VB,J#?

    Want to change something of a file?

    Have a nice day.


    Ghost,
    Call me ghost for short, Thanks
    To get the better answer, it should be a better question.

    Monday, July 09, 2012 1:14 PM
  • The question is: what is the function of a pinned field, in the standalone signature table, which is not documented in the ECMA-335 specification?
    Monday, July 09, 2012 5:02 PM
  • So what is pinned field?

    Ghost,
    Call me ghost for short, Thanks
    To get the better answer, it should be a better question.

    Tuesday, July 10, 2012 3:32 AM
  • This is an interesting related thread you may want to take a look at: http://www.netframeworkdev.com/building-development-diagnostic-tools-for-net/field-signatures-in-standalonesig-table-30658.shtml

    Karel and Rick on that thread are both developers on the CLR team, so you can trust those responses. The short answer sounded like officially that metadata is invalid according to spec, however something is (erroneously) generating it anyways. If you are particularly curious I can see if Karel ever found an answer.

    Tuesday, July 10, 2012 9:16 AM
    Moderator
  • If you could Noah, I would appreciate asking Karel and/or Rick if they found anything else out.  If you resolve a signature using Module.ResolveSignature with the proper metadata token (0x11 << 24 | 0xXXXXXX, where XXXXXX is the index within the StandAloneSign table, and 0x11 << 24 is the StandAloneSig table identifier) you get the exact byte array that I've been parsing as a field signature, which starts properly with the 0x06 field prolog.  The .NET Framework doesn't seem bothered by retrieving the data, and even seems as though there's no metadata validation prior to retrieval.  I'm curious as to their function, because in enumerating the libraries of the BCL there are 3,388 fields in the x64 version of the runtime, and 3,474 fields in the x86 version of the runtime, in all of the StandAloneSig tables, of which, 12 are mysteriously marked as pinned (both x86 and x64).  Given their prevalence I think they're more than junk data.

    Given my outside view I can only speculate, but I figure in knowing their function/purpose, I could write a more complete compiler framework if my understanding is more accurate.  My intuition thinks that they're more than junk because if they are junk they're wasting space, and I think it would then violate the idea of having a compressed representation of the metadata.  A hand full can be viewed as junk, but a few thousand seems a bit excessive.

    Thanks,

    -Allen Copeland Jr

    Tuesday, July 10, 2012 7:53 PM
  • I forwarded this along to Karel so hopefully we can both get a better answer : )

    -Noah

    Tuesday, July 10, 2012 11:11 PM
    Moderator
  • Noah,

    After messing around further, and noticing that even the C# compiler emits Field signatures in the StandAloneSig table, the signatures seem to relate to PDB debugging symbols.  When you emit symbols with the Debug or Release versions of your code, I'm guessing a StandAloneSig entry is injected and referred to by the PDB file.  If you are in release mode and you generate no PDB info, the StandAloneSig table contains no Field signatures.  One such condition for the emission of such information is constants within the scope of a method body.

    As far as that's concerned, I am guessing that one function of the fields in this table is to enable the IDE to provide feedback on constant values when a breakpoint is encountered.  Otherwise the IDE would have to parse the file, and resolve constants just to provide such information.  Without pdb symbolic information, breakpoints don't even work, and the information within the method body would never enter into context since you can't step into it.

    Granted this is circumstantial at this point, I haven't finished parsing method bodies, and I'm not sure where to begin with the debugging information since the CILDB format seems out of sync with the tooling provided by Microsoft (that is: they defined it, but don't consume the format, if this is wrong, let me know.)

    Thanks,

    -Allen Copeland Jr


    Thursday, July 12, 2012 5:27 PM
  • I followed up on your hunch about Field Signatures produced by C# (unpinned), it appears correct. The C# compiler is adding adding a Field signature in the StandAloneSig table for each const local variable definition. In the Pdb it encodes the name, the metadata token for that signature, and the value as a VARIANT. The API doesn't indicate this linkage back to the metadata is there, but looking at the code of the PDB reader I can tell that it is.

    The ILDB format defined by CLI is used by CLR only for in-memory symbol data for assemblies generated by reflection-emit that don't have the Save option specified. In other words we elected never write that format to disk. For all on disk symbol data we use the PDB format as a container and CodeView format inside, neither of which is documented. However there are public APIs which manipulate the format here: http://msdn.microsoft.com/en-us/library/ms232451

    Despite the lack of formal documentation, if you were to go look at the source for CCI (which is publically available) you would discover that it does crack the PDB file format directly and reads managed debug data. Implicitly you find out quite a bit about the format by examining that source.

    Thanks for taking the time to mess around and discover something interesting! I'm going to poke around internally to see what it would take to get the C# compiler to stop emitting this non-standard-compliant metadata. We'll still need to wait for Karel for info about the pinned field signatures you initially spotted, as it sounds like that is coming from a different compiler altogether.

    Thanks,

      -Noah

    Thursday, July 12, 2012 8:43 PM
    Moderator
  • If I had to make a guess, based off of the large chunks of data emitted by the compiler, I'm guessing it's from the C++/CLI compiler.  It generally makes a lot of use of pointer arithmetic, and it's possible they serialized datasets that the compiler needs to have marked as pinned purely so the type is defined as the constant was.  I'm thinking something similar to how the C# compiler handles array initialization, it's data declared at a certain location.  Beyond that I'm still at a loss, it seems that the C++/CLI compiler also emits similar field signatures in the StandAloneSig table when you declare internal_const values, but as for the magical combination that causes them to be, I can't say as I'm a hobbyist bore from the managed world (i.e. I don't use C++/CLI, I haven't had the need to, being a hobbyist.)

    Thanks,

    -Allen Copeland Jr.

    Friday, July 13, 2012 1:09 AM
  • IMAGE_CEE_CS_CALLCONV_FIELD (aka FieldSig = 0x6) in StandAloneSig is generated by C++/CLI - that was covered by the original thread http://www.netframeworkdev.com/building-development-diagnostic-tools-for-net/field-signatures-in-standalonesig-table-30658.shtml
    It was tracked down to C++ compiler, the fix was prototyped, but it broke VS debugger locals view. It was realized the bug fix has much bigger impact and would require changes across all compilers or change to PDB format and all PDB readers and writers (around ISymUnmanagedWriter::DefineLocalVariable API). The existing C++ bug was (correctly) closed, but new bug for PDB change was likely not created - at least I don't have any record for it :(.

    BTW: I was not aware of the fact that C# itself generates fields in StandAloneSig table. That is news for me - the email thread I have says C# should have stopped emitting such entries after .NET 2.0. It's probably a sign it is even more complicated issue than we understand.

    I can't find any known issue about ELEMENT_TYPE_PINNED in such field signatures.

    -Karel

    Tuesday, July 17, 2012 6:18 AM
    Moderator
  • Karel,

    Thanks for your insight.  I kind of figured that due to the link between the locals and the debugging symbols generated that the link went pretty deep.  My guess is the dependency has been in there a lot longer than anyone had anticipated.  The C++/CLI compiler emits the information whether the library is marked as debug or release, though the number of emitted entries changes, my guess is there's a much tighter dependency on the table than anticipated, the same goes for the C# compiler.  Hopefully as I research this further I might be able to figure out how to cause the C++/CLI to emit the pinned fields.  I only assume they're pinned because the signature contains the proper prolog for pinning, the signature's doesn't parse otherwise.  If you're interested I could give a signature index relative to a specific version of a framework library that emits a 'pinned field' signature within the table.

    Thanks,

    -Allen Copeland Jr.

    PS: The irony of finding C# emits field signatures in the StandAloneSig table is the clue involved the constant type-names I used in parsing the various signatures within the metadata.  They appeared sparsely throughout the code, and made it easier to track down.
    Tuesday, July 17, 2012 9:35 AM