I need official PST and OST binary file format specs

Question

I have an interop scenario. I have OST and PST files that contain emails and documents that cannot be altered. Think medical records and legal records. I need to be able to access these directly in the files without altering them in any way.

When will these file format be available?

Answer 1

Hi ODO_ODO,

Someone from y team will be contacting you shortly regarding this question.

Thanks and regards,

---

SEBASTIAN CANEVARI - MSFT Senior SEE Protocol Documentation Team

Answer 2

The PST file format has been asked for before. Microsoft appears unwilling to document this one.

the OST file format appears to be in a similar state.

I guess your options for PST are:
1. Use some kind of Microsoft-proprietary tool.
2. Use libpst
3. Write your own tool based on the (incomplete) documentation in libpst and whatever you can extract by similarity in MS-OXPROPS.

libpst is available at http://www.five-ten-sg.com/libpst/

For OST, there is presumably a Microsoft-proprietary tool, but I'm not familar enough with them to even point you to something.

There also appears to be some new FOSS work going on to look at OST, in libpff. Check http://sourceforge.net/projects/libpff/ for both sample code and some documentation. I haven't tried this one, but there aren't a lot of good options.

Hope that helps.

Brad

Answer 3

Brad,
Thanks. The info you provided is helpful. However I need official documentation so that I can write code that I am certain extracts documents without error, omissions or altering data. I don't believe I will be able to get away with tools based on partial or reverse engineered specifications. Like PPT, XLS, DOC I need official PST and OST documentation. EMAIL retention requirements, medical documents, legal documents are just some of the scenarios that could present themselves. Microsoft tools proprietary or otherwise are not useful since I need to be able to work with these documents in a *nix system or possibly a mainframe. Then I need to tackle Access database files but that is another story for another thread.

Answer 4

Hi ODO_ODO,

I am the engineer who has taken ownership of your question. I will update this thread as things progress.

Dominic Salemno
Senior Support Escalation Engineer

Answer 5

Any update on this? I have strategic projects in limbo.

Answer 6

ODO_ODO,

 

Currently we do not have any plans to document the .PST or .OST file formats. However, utilizing Stephen Griffin’s MFCMAPI allows one to navigate the message store which includes local PST files. You will find the executable and source code of mfcmapi as a good starting point (http://mfcmapi.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=21390). However, if you do not wish to install the Outlook client another option would be to install the Microsoft Exchange Server MAPI Client and Collaboration Data Objects 1.2.1, which contain the Exchange implementation of the PST provider (MSPST32.DLL). The Exchange PST provider only supports ANSI. If you need support for UNICODE, you will need the version that ships with Microsoft Outlook.

 

Is there anything else that I may assist you with?

Dominic Salemno
Senior Support Escalation Engineer

Answer 7

Dominic,
As I mentioned above, I need to be able to access these files on *nix and possibly mainframe based computer systems. I don't have an exact total but there are easily more than a terrabyte of archive files that could be as old as 15 years old. Even Microsoft does not convert these PST files but instead provides backwards access to Office 97 versions of the PST files in Office 2007. I understand that the future direction for Microsoft may include storing Office files in OpenXML or ODF format but it is completely impractical or impossible to go back and convert these old PST files even if the conversion utilities become available. There are PST files that contain, but not limited to, medical memos and records, legal memos and emails, government memos and emails, executive memos and emails, files under document retention rules and much more. Some of these PST files have been offline for many years. I do believe that the intent of the PST file is to be an offline archive file. A primary function of an archive is to hold information in an accessible, persistent and immutable form for long periods of time. At least that is what I understand the Microsoft documentation to state. I don't know if you have ever been required to testify under cross examination but to have to state that information was accessed through unofficial, reverse engineered information or hackers tools is not something I want to defend. That immediately brings into question the completeness and accuracy of the information.

This is my understanding of your replies. Please let me know if I have this wrong.

Microsoft is either unwilling or unable to document the PST and OST archive file formats. Even though other file formats such as XLS, DOC and PPT have been fully documented under the Open Specification Promise, OST and PST have been intentionally excluded. The only recommendation that Microsoft can make to access the data in these archive files is to either install Microsoft products or code which is not possible on Unix based machines or to rely on unofficial and reverse engineered 3rd party tools and information. Past, present and future use of these archive files will require lock-in to Microsoft products to be able to access the archived data.

I look forward to your reply.

As a heads up, we are about to go down this path again for other Office file formats that have been used or recommended for storing scans, images, documents and databases. If companies like Adobe are willing to do this for files like PDF files I don't understand why Microsoft is taking this posiiton with their essential Office files.

Answer 8

Well it has been another month and I still have strategic projects in limbo. It looks like I am not going to get an answer out of you so I am going to have to escalate.

Answer 9

Were you able to obtain these specs?

Answer 10

Greetings,

Please review the following regarding the .PST File Format Specification:

http://blogs.msdn.com/interoperability/archive/2009/10/26/roadmap-for-outlook-personal-folders-pst-documentation.aspx

Dominic Michael Salemno
Senior Support Escalation Engineer
US-CSS DSC Protocols Team