Circular Reference in the TNEF spec?

Thu, 15 May 2008 00:42:13 Z

Section 2.4 Encapsulated Message Properties

This looks like a couple of circular references to me:

MsgPropertyContent = PropertyScalarContent / PropertyMultiScalarContent / PropertyMultiVariableContent

PropertyScalarContent = MsgPropertyContent [PropertyPad]

PropertyMultiScalarContent = PropertyContentCount *MsgPropertyContent [PropertyPad]

I can't find an explanation for this type of encoding in the ABNF rfc and I can't think of how to implement it. Any explanation would be appreciated.

Greg

Circular Reference in the TNEF spec?

Thu, 15 May 2008 14:55:32 Z

Hi Greg,

Thanks for your question. I have moved your posting to the Windows Protocols forum which is the better forum for this question.

Circular Reference in the TNEF spec?

Tue, 20 May 2008 22:54:24 Z

I should also like to point out that on page 22, the comment indicates that Boolean types are 16 bits but examining TNEF data leads me to believe they are actually 32-bit.

Circular Reference in the TNEF spec?

Wed, 21 May 2008 19:22:21 Z

2.4 Encapsulated Message Properties

; Only present when MsgPropertyId is > 0x8000

NamedPropSpec = PropNameSpace PropIDType PropMap

Should be:

; Only present when MsgPropertyId is >= 0x8000

Circular Reference in the TNEF spec?

Wed, 21 May 2008 19:38:30 Z

Page 12

2.2 ABNEF Description

;Attachment RendData

idAttachRendData = %x02.02.06.00

Should be:

idAttachRendData = %x02.09.06.00

Circular Reference in the TNEF spec?

Thu, 29 May 2008 19:51:34 Z

Section 2.4 Encapsulated Message Properties

page 21

; Used if PropIDType is IDTypeString. Contains a length, then a

; UTF-16LE encoded UNICODE string ; and terminating 2-byte zero character, padded to UINT32 boundary

PropMapString = UINT32 *UINT16 %00.00 [PropMapPad]

Should be clarified as:

; Used if PropIDType is IDTypeString. Contains a length, then a

; UTF-16LE encoded UNICODE string. The length includes the terminating 2-byte zero character. Optional padding to UINT32 boundary

Circular Reference in the TNEF spec?

Thu, 29 May 2008 19:56:50 Z

Section 2.4 Encapsulated Properties

page 24

This is completely wrong:

PropertyVariableContent = MsgPropertySize MsgPropertyValue [PropertyPad]

MsgPropertyValue is defined at the top of the section (page 21) as:

MsgPropertyValue = MsgPropertyTag MsgPropertyContent

MsgPropertyTag = MsgPropertyType MsgPropertyId [NamedPropSpec]

Variable length types do not include the MsgPropertyTag again. These just contain the data for the specified length.

Circular Reference in the TNEF spec?

Wed, 11 Jun 2008 19:33:36 Z

I'm getting pretty excited. My original post was almost a month ago which means it might be getting a response any week now! Woohoo! Just in time for me to ship my reader.

For anyone else out there building a TNEF reader, it is possible to do using the MS-OXTNEF spec. Sort of.

Use the errata listed above, of course, but there are little interesting things like nested messages that you'll just have to pick through the TNEF data using a binary editor to get right.

Some of the property types (strings) always seem to have a multi-value count, other do only for MV types.

In short, the spec is an interesting and helpful starting point for building a TNEF reader/writer but don't go just by the written word. Look at TNEF output from different versions of Exchange.

Circular Reference in the TNEF spec?

Thu, 12 Jun 2008 16:09:41 Z

Greg,

Sorry this got missed as your orginal post got moved to the wrong forum and just plain fell under the radar. I've moved it to the Exchange Protocol Forum so it will receive the proper attention. I'm taking your feedback and filing bugs against the documentation.

Thank you for your feedback.

Circular Reference in the TNEF spec?

Thu, 12 Jun 2008 16:24:02 Z

Greg, Booleans and I2 are 16-bit and all values must be padded to 4 bytes which is what you are seeing in the stream.

Circular Reference in the TNEF spec?

Thu, 12 Jun 2008 16:29:28 Z

Gregory K. Miskin wrote:

2.4 Encapsulated Message Properties

; Only present when MsgPropertyId is > 0x8000

NamedPropSpec = PropNameSpace PropIDType PropMap

Should be:

; Only present when MsgPropertyId is >= 0x8000

Bug has been filed against the documentation.

Circular Reference in the TNEF spec?

Thu, 12 Jun 2008 16:33:51 Z

Gregory K. Miskin wrote:

Section 2.4 Encapsulated Message Properties

This looks like a couple of circular references to me:

MsgPropertyContent = PropertyScalarContent / PropertyMultiScalarContent / PropertyMultiVariableContent

PropertyScalarContent = MsgPropertyContent [PropertyPad]

PropertyMultiScalarContent = PropertyContentCount *MsgPropertyContent [PropertyPad]

I can't find an explanation for this type of encoding in the ABNF rfc and I can't think of how to implement it. Any explanation would be appreciated.

Greg

Scalar content has a single value and a pad if needed, to reach a 4 byte boundary. Multi-scaler has a count padded to 4 bytes, followed by that number of properties, each padded to 4 bytes. Varaible content is a count, then that number of properties in the form size, value, pad.

Does this help?

Circular Reference in the TNEF spec?

Sun, 15 Jun 2008 01:11:23 Z

Tom Devey - MSFT wrote:

Gregory K. Miskin wrote:

Section 2.4 Encapsulated Message Properties

This looks like a couple of circular references to me:

MsgPropertyContent = PropertyScalarContent / PropertyMultiScalarContent / PropertyMultiVariableContent

PropertyScalarContent = MsgPropertyContent [PropertyPad]

PropertyMultiScalarContent = PropertyContentCount *MsgPropertyContent [PropertyPad]

I can't find an explanation for this type of encoding in the ABNF rfc and I can't think of how to implement it. Any explanation would be appreciated.

Greg

Does this help?

Tom,

Thanks for the reply.

I grok the padding and count. I was trying to point out that there is a circular reference for MsgPropertyContent, that is, you can't get to the end of its definition.

In other words, in a simplified form, MsgPropertyContent == PropertyScalarContent == MsgPropertyContent....

There's no real definition of PropertyScalarContent and PropertyMultiScalarContent. You have to, in my opinion, define the scalars and not assume anything.

Greg

Circular Reference in the TNEF spec?

Tue, 17 Jun 2008 20:08:35 Z

Thanks, I will pass the feedback to the documentation group.

Circular Reference in the TNEF spec?

Tue, 24 Jun 2008 02:28:29 Z

Section 2.3.3 attFrom

There is no mention of padding here. Clearly padding is taking place between the end of sender-email-address and the checksum. The padding is the delta between structure-length and (8+ sender-name-length + sender-email-length) . Maybe it isn't padding but there are NULL bytes there that cannot be accounted for with the string lengths. In one particular case it was 8 bytes.

I can't account for these bytes in the spec and am assuming they are padding.

Circular Reference in the TNEF spec?

Thu, 26 Jun 2008 16:36:22 Z

Greg, I've emailed the document owner for clarification.

Developer Consultant

Circular Reference in the TNEF spec?

Fri, 27 Jun 2008 18:25:54 Z

Greg,

The version 1.0 of the Exchange Server Protocol documents released today. The top-level URL for the documents can be found here http://msdn.microsoft.com/en-us/library/cc307725(EXCHG.80).aspx. You can also download the zip file that contains the PDF version of the Exchange Server Protocol documents http://go.microsoft.com/fwlink/?LinkId=115073

For your interest the updated [MS-OXTNEF] document can be downloaded at http://download.microsoft.com/download/5/D/D/5DD33FDF-91F5-496D-9884-0A0B0EE698BB/%5BMS-OXTNEF%5D.pdf

From the bugs you filed:

"MsgPropertyId is > 0x8000" now correctly reads "MsgPropertyId is >= 0x8000"
Boolean types are 16 bits but examining TNEF data leads me to believe they are actually 32-bit.

The documentation has been updated to added the follow:

PropertyScalarContent = MsgPropertyContent [PropertyPad]

; PropertyPad – between 0 and 3 zero-filled bytes, added to the streamed
; property values to achieve 4-byte boundary. Writers MUST use zero bytes
; and Readers MUST permit non-zero bytes. These pad bytes MUST be counted
; in the checksum of the containing attribute.

PropertyPad=*3ZERO
ZERO= %x00

Section 2.2 ABNF has been updated
Section 2.4 Encapsulated Message Properties updated

;Used if PropIDType is IDTypeString. Contains a length, then a
; UTF-16LE encoded UNICODE string
; The length includes the terminating 2-byte zero character. Optional padding ; to UINT32 boundary
Section 2.4 Encapsulated Properties
"PropertyVariableContent = MsgPropertySize MsgPropertyValue [PropertyPad]"
now reads
"PropertyVariableContent = MsgPropertySize MsgPropertyContent [PropertyPad]"

Note that the "Section 2.3.3 attFrom" and the "Cirular reference issue" will be updated in the next round of protocol document updates.

If you have any questions or find any discrepancies in the new documentation please don’t hesitate to post questions or feedback to this forum.

Again thank you for providing valueable feedback.

Developer Consultant

Circular Reference in the TNEF spec?

Wed, 02 Jul 2008 20:56:12 Z

I talked to the document owner yesterday in regards to the Message Properties Circular Reference and he made some changes in the version 1.0 release that should address this issue.

Please let me know it this doesn't address your issue so I can follow-up with the document owner to make sure it does get fixed.

Circular Reference in the TNEF spec?

Wed, 02 Jul 2008 22:56:33 Z

Page 23:

MsgPropertyContent = PropertyScalarContent / PropertyMultiScalarContent / PropertyMultiVariableContent

; Scalars – Types Int16, Int32, Flt32, Flt64, Currency, AppTime, ; Bool, Int64, Systime, CLSID ; The data for the particular property is written to the stream and if necessary, ;padded with bytes (which SHOULD be zero) to achieve a multiple of 4-bytes in length.

PropertyScalarContent = MsgPropertyContent [PropertyPad]

This is still a circular reference until you define PropertyScalarContent without reusing MsgPropertyContent. So it should read:

PropertyScalarContent = TypeInt16 / TypeInt32 / TypeFlt32 / TypeFlt64 / Type Currency / TypeAppTime / TypeBoolean / TypeInt64 / TypeSystime / TypeCLSID [PropertyPad]

See the difference? I can now get to the end of the definition of MsgPropertyContent. The same thing needs to be done with PropertyVariableContent but I don't want to do all the work here. :)

Circular Reference in the TNEF spec?

Thu, 24 Jul 2008 16:48:33 Z

"Section 2.3.3 attFrom

There is no mention of padding here. Clearly padding is taking place between the end of sender-email-address and the checksum. The padding is the delta between structure-length and (8+ sender-name-length + sender-email-length) . Maybe it isn't padding but there are NULL bytes there that cannot be accounted for with the string lengths. In one particular case it was 8 bytes.

I can't account for these bytes in the spec and am assuming they are padding."

The feedback from development is it doesn't matter since the decoding will work fine with or without the pad. In reviewing the documentation development is looking at putting verbage to clarify this even though it should work fine either way.

Thanks for the great feedback.

Developer Consultant

Circular Reference in the TNEF spec?

Mon, 22 Jun 2009 13:37:19 Z

Hi Tom,

Even though this thread is a bit old - I found it relevant as the documentation still hasn't been corrected.

I'm using MS-OXTNEF v2.0 released April 10, 2009 - but the circular reference issue with PropertyScalarContent [page 24] still exist.

Also, the "Section 2.3.3 attFrom" issue may also not have been addressed(?) I haven't verified myself as of yet whether or not padding exists - but if it does as Gregory mentions, then the documentation doesn't appear to reflect this.

/Michael

Circular Reference in the TNEF spec?

Fri, 10 Jul 2009 22:49:36 Z

A quick review of the orginal Technical Document Issue (TDI) that I filed it was supposed to have been fixed with the August documentation release of 2008. Reviewing both Exhange 2007 and the Exchange 2010 [MS-OXTNEF] documentation it was never fixed. I created a new TDI to get this fix.

Thank you for your feedback.

Developer Consultant

Circular Reference in the TNEF spec?

Fri, 10 Jul 2009 23:05:32 Z

As for Section 2.3.3 attFrom there was an end note that was added to reflect the differences for the "pad".

Hope this helps.

Developer Consultant

Circular Reference in the TNEF spec?

Wed, 15 Jul 2009 08:17:24 Z

Hi again Tom,

I just discovered a minor issue in the documentation page 11 - the property idParentID seems to have been removed between v1 and v2 of the document.

It should read:

idMessageAttr = idMessageClass / idFrom / idSubject / idDateSent / idDateRecd / idMessageStatus / idMessageID / idParentID /

idConversationID / idBody / idPriority / idDateModified / idRecipTable / idOriginalMessageClass / idOwner / idSentFor /

idDelegate / idDateStart / idDateEnd / idAidOwner / idRequestRes

; PidTagParentKey

idParentID = %x0A.80.01.00

I found the problem after running a forward rule (in Outlook2007 on an Exchange 2003)

/Michael

Circular Reference in the TNEF spec?

Thu, 16 Jul 2009 17:37:19 Z

Michael,

I have filed a Technical Document Issue (TDI) to get this fixed. Thank you for the continued feedback on this document.

Developer Consultant