[MS-OXCPRPT] Semantics for stream region locking

Tue, 07 Apr 2009 09:55:51 Z

G'day long suffering forum people,

I'm looking at MS-OXCPRPT, Section 2.2.23 "RopLockRegionStream" and Section 2.2.24 "RopUnlockRegionStream".

Is it possible to get some more information on the semantics of these operations? In particular:
1. What are the allowed sequences for locking/unlocking? For example, if I lock the range 0-1023, and then lock the range 1024-2047, is it legal to unlock the range 0-2047 in one operation? Is it legal to unlock the range in several sections (e.g. 0-511 then 512-1023)? What happens to the lock if I unlock a region greater than is locked (e.g. unlock 512-4095)?
2. What is the meaning of "owner" in 2.2.23.1.1 "Lock Flags"? That is, if I lock a region, who is allowed to read/write to it? Anything with the same session?

Thanks again,

Brad

[MS-OXCPRPT] Semantics for stream region locking

Wed, 08 Apr 2009 04:52:52 Z

Hi Brad,

I have done my research on your questions above and I'm waiting for feedback from development

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Wed, 08 Apr 2009 04:57:05 Z

Tom,

Once again, thanks for your work on this.

Much appreciated.

Brad

[MS-OXCPRPT] Semantics for stream region locking

Thu, 07 May 2009 18:00:03 Z

Hi Brad,

You can have a single lock per region in a stream that is defined by a start and an end with a defined LockFlags. You can have multiple regions locked in a stream but they cannot overlap. Each lock is treated as separate entity and must be unlocked as a single entity unless the lock has been allowed to expired. If you attempt to unlock an expired lock the ReturnValue in the Response buffer will be error code MAPI_E_NETWORK_ERROR (0x115).

In #1 above if you would have two separate locks that would require two separate unlocks with matching LockFlags. If you lock a stream 0-1023 and then unlock the stream from 0-511 the lock 0-1023 will be cleared. If you lock a stream 0-511 and then unlock the stream 0-512 the stream will not be unlocked and no error code will be returned.

As for question #2 the use of the language around Lock Flags ‘owner’ because the check is really from what I see (unless I’m missing something obvious) just a simple match.

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Thu, 07 May 2009 22:33:44 Z

Tom,

That is probably the answer to my original question, but your answer raises a few more issues :-) Following along from the original questions:

3. I don't see anything in MS-OXCPRPT about locks "expiring". Can you tell me more about this? For example, is it based on time, or something else (number of ROPs, perhaps something else)? Is it just a server side setting that applies to all users? Is there some client side control over it?

4. From your description, if I have a lock starting at RegionOffset of size RegionSize, then I will reliably unlock if I use the same RegionOffset and RegionSize (and flags) in the unlock. If I don't use the same RegionOffset, it will fail. If I use a RegionSize in the unlock that is less than or equal to the RegionSize in the lock, it will unlock; howevr if RegionSize in the unlock is greater than the RegionSize in the lock, it will return success but not actually unlock. In pseudo-code:

process_unlock( uint64_t unlockRegionOffset, uint64_t unlockRegionSize, uint32_t unlockLockFlags)
{
        // find the right lock
        foreach ( streamLock, thisStream ) {
                if ( unlockRegionOffset == streamLock.RegionOffset ) {
                       // this is the lock we're wanting to unlock
                       if ( ( unlockRegionSize <= streamLock.RegionSize ) &&
                            ( unlockLockFlags == streamLock.LockFlags ) ) {
                               unlock( streamLock );
                               return MAPI_E_SUCCESS;
                       } else {
                               // fail silently
                               return MAPI_E_SUCCESS;
                       }
                 }
          }
          // we couldn't find a matching lock
          return MAPI_E_NETWORK_ERROR;
}

Does that roughly match your understanding?

Brad

[MS-OXCPRPT] Semantics for stream region locking

Fri, 08 May 2009 05:11:16 Z

Brad,

Note that that there may be a scenario where there may be an "owner" so I'm looking for some addition feedback from Exchange development.

I was re-reading the documents after I posted the response above and yes timeouts are not mentioned. Looks like a opportunity for some additional details.

In reviewing your code and looking at how Exchange treats the unlock I realized that my comments above about unlock is incorrect. Turns out unlock does support the ability to unlock multiple locks on a stream with one request. As an example if one has locked regions in a stream of 50-500 and 512-1023 one could pass in the start of 0 and the offset of 1024 and both regions would be unlocked. If one or more of the locked regions locks had expired then MAP_E_NETWORK_ERROR is returned.

One additional observation is if the offset and size passed to unlock a stream defines a region that contains no locked regions then no error is returned (MAPI_E_SUCCESS).

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Sat, 20 Jun 2009 03:42:47 Z

Brad,

Ok I was able to get some clarification of the "owner". When you call RopLockRegionStream, one of the parameters is an InputHandleIndex, which points to a server object handle (that refers to the stream you are operating on). If the lock is granted, then exclusive write access (or read/write access, depending on LockFlags) will be given to the user of that specific server object handle which is now termed the "owner". If another handle referring to the same stream object cannot write to the stream, even if that handle is on the same session.

Only the handle used on the RopLockRegionStream has access.

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Sat, 20 Jun 2009 04:02:34 Z

Tom.

Where you wrote " If another handle referring to the same stream object cannot write to the stream, even if that handle is on the same session.", should that be " Another handle referring to the same stream object cannot write to the stream, even if that handle is on the same session."?

Also, the part about timeouts and locks expiring is still not clarified.

Thanks for continuing to follow this up.

Brad

[MS-OXCPRPT] Semantics for stream region locking

Tue, 23 Jun 2009 22:07:48 Z

Yea your correction is correct. Ah looking at what I asked development I missed the ask about timeouts. I will follow-up with development for clarification. Just to make sure I frame the question correctly what additional information are you looking for outside of the fact that locks have timeouts and can expire and return an error.

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Wed, 24 Jun 2009 08:34:22 Z

Tom.

I'm not really sure what the exact question is, because I don't know more than "what is this timeout business".

To help the discussion:
1. Under what circumstances can locks time out?
2. What is the timeout criteria (e.g. is in seconds, or in operations or bytes or something else?)
3. What does the server do when the lock times out? In particular, what error code represents a timeout?
4. What is the client expected to do in response to a timeout?

Sorry to be so vague.

Brad

[MS-OXCPRPT] Semantics for stream region locking

Thu, 25 Jun 2009 00:25:38 Z

I passed my feedback on what I thought was important to document and I added your suggestions to the discussion with development.

Thanks!

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Wed, 29 Jul 2009 22:57:25 Z

Hi Brad,

This is moving forward on my side with development. More details on the missing content will be posted soon here on this thread.

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Sat, 08 Aug 2009 05:30:24 Z

Hi Brad,

The development for these ROPs was done in the mid 1990’s originally for Schedule+ and the the stream lock/unlock were last used by Outlook 2000. They are not used by subsequent versions of Outlook. The ROPs were added to the documentation because the current version of Exchange 2007 Load Generator sends the ROPs over the wire. There is a bug being filed to have them removed. There will be additional information added to the document MS-OXCPRPT in a future release.

As for your questions here is the feedback from development:

1. Under what circumstances can locks time out?

Region locking only has an effect on streams on folder objects. Streams for message or attachment objects are independent of each other, so the locks have no effect on them (you cannot lock yourself out of a region, and your lock doesn’t affect any other version of the stream). This is limited to folder streams. During a LockRegion call for folder streams, the server will first check for existing locks on that region of the stream. If any are found, we will query the last activity time on the session that locked that region. If the last activity is greater than a certain interval, then that lock is marked as expired.

2. What is the timeout criteria (e.g. is in seconds, or in operations or bytes or something else?)

It is defaulted to 30 seconds. There is a way to override the default timeout in the registry but the override key has never been published.

3. What does the server do when the lock times out? In particular, what error code represents a timeout?

All pending stream changes are discarded. Any read/write/setsize/commit that would encounter the region that has the expired lock will fail with the MAPI_E_NETWORK_ERROR.

4. What is the client expected to do in response to a timeout?

The burden is put on the client and effectively you have retry logic. When an expired lock is encountered, unlock region will fail with the same error as mentioned above, except that it will also remove the expired lock.

There is no way to remove expired locks outside of trying to unlock the lock. There is a race condition with this because when one would attempt a recovery which would consists of “a) unlock the region that contains expired locks b) lock the region myself", then two clients performing the operation could have one client performing step “a” after the other did step “b”, thus removing valid locks. The retry logic in Outlook 2000 ‘s Schedule+ was based on user input, which made it very unlikely that two clients would recover at exactly the same time and hit these problems.

Hope this helps,

Developer Consultant

[MS-OXCPRPT] Semantics for stream region locking

Sat, 08 Aug 2009 11:34:22 Z

Tom,

I think I see why this one took a while :-)

Thanks for the work on this one. Can you also please pass my thanks the devel team who worked on this?

Brad