Adding a recipient to the .msg file

Question

Hello!

I've a task to develop an utility that could write an msg file from scratch without using any mapi libraries. This limitation is strict, unfortunately.

So I've studied the structure of the compound file, the red-black trees, etc.

Now I'm trying to make a file with a message. I can manage every stream and storage like __properties_version1.0, recipient storages and add data to them. I can build a draft message file without any recipients. What I need to do is to "attach" a node to this streams tree that would be a storage for the recipient. I can make a small 7 node storage with needed streams like __recip_version1.0_#0000000, streams that store PidTagEntryId, PidTagDisplayName and so on. Moreiver, I set the value of the top-level stream that is responsible for the DisplayTo form. However, when I open the file and move the mouse to the email address in the "To:" form, the recipient's email disappears. I suppose that I've missed something when I was adding the recipient container (a red-black tree with 7 nodes) to the global tree.

I'd like to find out what actions should be made when the recipient is added to the message.
I've already contacted Dmitry Streblechenko from Microsoft whose comments on the net about outlook technologies are very useful, but he had no idea.

Answer 1

Hi Mx,

I will have to debug this one to track down why it does not work.  Can you please upload the .msg file to
https://sftus.one.microsoft.com/ChooseTransfer.aspx?key=269d88c6-9811-4edf-9639-5d055d42cd9b

Password: kC{kz)]xr]*

Thanks!

---

Developer Consultant

Answer 2

Hi Tom,
I've sent you my .msg file! Thanks for help

Answer 3

I got the file.  Thanks!

---

Developer Consultant

Answer 4

I've found the problem and the solution. The thing was that I wrote two different pices of data into the RecipientType field. I was confused by the MS-OXOMSG document where it is said that this field: "Represents the recipient type of a recipient on the message. This property MUST be set on each recipient. The format of this property is listed in the following " (then go the Originator, the Primary Recipient, Cc and Bcc). But "the flags in the following table can be combined with the values listed in the previous table. " and then two more flags go.
So these additional flags must have not been set

Answer 5

Unfortunately, if I add a large attachment and the overall .msg file size is over 109*128*512 = 6.9 megabytes, the attachment is unreadable. Maybe there is something wrong with the DIF structure.
Tom, may I send you the new version of my file? Could you take a look at it?

Answer 6

Please upload the file to the location I mentioned above.  I have a lot of open customer questions I'm working on so my first chance to look at this will be Thursday.

---

Developer Consultant

Answer 7

Oh, I've managed to find the mistake.
It seems that now everything works fine! I couldn't imagine it is possible to do such a thing without the usage of any MAPI stuff...

Answer 8

Thanks for the update. 

What things should be included in the protocol documentation that would have better helped you?

---

Developer Consultant

Answer 9

Well, I was confused by the lack of examples of the red-black tree in the .msg file. The documentation on compound files is OK, but an example of a valid tree of streams from a .msg file would be great. I guess that's the trickiest point in my task.
Then, the order of information in the file. For some reason after opening my file Outlook changes the order of some sectors in FAT and DIF structures. Maybe due to the wrong order of data in the file it cannot be edited in Outlook (an error occurs - "The message interface has returned an unknown error. If the problems persists, restart Outlook"). I'm happy that this error is not important now!
And, of course, the trick with the RecipientType field, which I described a few posts earlier, was REALLY embarrasing. I think that it should be mentioned that some flags must not be set in .msg files since they are of no use for Outlook. It was sad to discover that just one bit in the whole file ruined everything

Answer 10

Damn, Outlook 2007 does not want to open it... Tom, I am terribly sorry for asking you again to take a look at the file. Can you give me a new password? Seems the battle is not over yet=(

Answer 11

Here is a new workspace to upload the file to:

https://sftus.one.microsoft.com/choosetransfer.aspx?key=ddf77ad9-6d4e-4e66-9714-df12dc472b1f

Password: s1$lUAXk9mg%

 

---

Developer Consultant

Answer 12

have sent it, thanks

Answer 13

It seems that Outlook 2007 adds streams that are of STGTY_INVALID = 0x0 type. My code generation class parsed the correct file and the output was

entry = new DirectoryEntry();
entry.Type = SID_Type.STGTY_INVALID;
entry.GUID = new byte[]{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
entry.Name = ""; /**/
entry.Color = SID_Color.DE_BLACK;
entry.SID = (uint)17;
entry.LeftChildSID = (uint)SectInfo.FREESECT;
entry.RightChildSID = (uint)SectInfo.FREESECT;
entry.RootChildSID = (uint)SectInfo.FREESECT;

maybe it fills free space within the Directory sectors with such streams

Answer 14

I got the uploaded file sub_u.msg and I got feedback from Outlook development today that should help me better scope this.

---

Developer Consultant

Answer 15

So, Tom, is there any update?

Answer 16

Hi,

Even I have a task to develop an library that could write an msg files from scratch without using any mapi libraries. Could you please guide how to do this? It will be greatly helpful if you can provide some sample in C#.

Thanks,

Amir

Answer 17

I am talking to the Outlook developer support team to help me out as it's not clear to me why it is not working.

---

Developer Consultant

Answer 18

Tom, thank you very much, I really hope the support team members will find what's wrong. I don't know why Outlook 2007 does not open any message file.
Amir, it is a great volume of work. What you need to do first is to read documentation on how compound files are organized (like MS-CFB http://msdn.microsoft.com/en-us/library/cc546605.aspx). Then after you implement all these FAT, DIF, MiniFAT, Ministream, Directory and other stuff, you switch to MS-OXMSG (http://msdn.microsoft.com/en-us/library/cc463912.aspx) and learn the pecularities of .msg files like Nameid storage, properties stream, etc.
Download this library I found on the net - http://ifolder.ru/11770665. It will help you to understand the basics of the compound file
My method of work was based on using this library. I made classes that encapsulate the functionality of compound file elements like DirectoryEntry or FAT. Then I wrote a bit more code in this library so that I could pass the path to an existing .msg file made by Outlook and the library would parse it and make a .txt file with a large C# method inside that builds the same structure of the initial file using my classes.
Anyway, that's a very time-consuming task

Answer 19

Hi,

Thank you very much. 

Your guidance are really helped alot. One more help, Is it possible tell me how long it took approximately for you, Please.

Thank You!!!

Thanks,

Amir

Answer 20

Sorry I am still waiting on a response from development.   

I did talk to someone on the Developer Support Messaging team he suggested that you use Outlook to craft a similar message and save it to MSG format and then use DocFile Viewer to see an example of what the protocol document’s output should be and then change your code as needed.  He said that this is the best way to get something working against the .msg file format.

---

Developer Consultant

Answer 21

Mx,

I am the engineer who has taken ownership of your case. I am investigating your issue and will update you as things progress.

Dominic Salemno
Senior Support Escalation Engineer

Answer 22

Mx,

Could you send me your sample files to [e-mail]?

Dominic Salemno
Senior Support Escalation Engineer

Answer 23

Mx,

I do not have insufficient information to proceed. In the meantime, the information regarding this issue will be archived.

Dominic Salemno
Senior Support Escalation Engineer

Answer 24

Mx,

I am still investigating this issue. I will update this thread as things progress.

Dominic Salemno
Senior Support Escalation Engineer

Answer 25

Mx,

Which reference are you utilizing in regards to the following directory entry type:

--> STGTY_INVALID = 0x0

entry.Type = SID_Type.STGTY_INVALID;

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Upon reviewing the latest version of MS-CFB on the Open Specifications site (http://msdn.microsoft.com/en-us/library/dd942138(PROT.10).aspx), you will see the following Object Types (1 byte):

Unknown or unallocated = 0x00
Storage Object = 0x01
Stream Object = 0x02
Root Storage Object = 0x05

The directory entry type you indicate is either of an unknown type or unallocated per Section 2.6.1 "Compound File Directory Entry" of MS-CFB on Page 24.

Does this information assist you?

Dominic Salemno
Senior Support Escalation Engineer

Answer 26

sorry guys for not responding to your posts, I was away for a long time... Dominic, thanks for your assistance, but I'm afraid our customer didn't like our project, so there is nothing to worry about now. As I remember, I didn't find out why Outlook 2007 makes such STGTY_INVALID entries, so I just made them too and it worked. I was just curious why these entries are needed by Outlook 2007. Nevertheless, it is over now =) Thanks for you post, anyway