locked
PortSet behaviour and OperationsPort RRS feed

  • Question

  • If PortSets are just a collection of independently handled ports and arbiters like Interleave deliberately round-robin receivers (see this post) , then how does DSS guarantee that (say), the notifications raised by service A are handled in the same order by some subscribing service B?

    For example, if A has a Replace and Update operation in its operations port and receives a Replace (which it then signals), followed by an Update (which it also signals), what's to prevent subscribing Service B actually processing the Update notification before the Replace?

    Or are there no such guarantees and its up to the messages themselves to carry some additional ordering information?

    Thanks
    Nick.

    ps, I am assuming that the receivers for these operations are within the correct receiver groups (i.e. exclusive) within their respective interleave.
    Wednesday, December 10, 2008 12:06 PM

Answers

  • Nick, you are correct that we will not preserve order across subscriptions, especially, across different message types, since they arehandled by different ports on the receiver end.
    Note however, that if the subscriber keeps track of the last message received timestamp, it can disregard any notification that was sent before that message (this indicates out of order). In the case of state modifying verbs, that rule can apply mostly to Replace: If you receive a replace, ignore any other notification (insert, update, Replace) that originated before it, its out of date.

    Our sopa envelopes include a timestamp, but its actually safer to either include a custome header (that is IDssSerializable) with a timestamp that is meaningful to your app, or include a DateTime field in the body.

    as you are well aware, while we can provide ordered notifications, we will do so at the expense of concurrency (since guaranteeing order across two independent entities means somehow introducing data dependencies between messages, manifested as a little send/receive/ack protocol)

    thanx
    g
    • Marked as answer by Trevor Taylor Wednesday, December 31, 2008 9:56 PM
    Tuesday, December 30, 2008 9:28 PM

All replies

  • Nick

    Based upon what I know about ports and interleaves I am not sure there is any guarantee of order, simply because there isn't any guarantee of order across ports. So PortSet or not, if they are different messages and arrive on different ports within the subscriber then order is not necessarily preserved.

    With that being said, I think that your hint at the use of exclusive groups and interleaves could setup a situation where it would happen. So its a solution that can be programmed in but its not in there by default.

    Bryan

    (Your ref post link is broken, I wonder if its due to the recent forum update)

    Wednesday, December 17, 2008 1:38 PM
  • Thanks Bryan,

    But if the interleave round robins, then the exclusivity won't necessarily provide the ordered behaviour - it will constrain the number of tasks that can run, but not which one does run.

    The concern I have is that for services that expose collection state, the symmetric subscriber model breaks down.

    If I have a collection of some entity T, whose operations port supports (say) Replace and Insert, then is there a possibility that three operations issued in quick succession

    Replace (with empty)
    Insert (item 1)
    Insert (item 2)

    could, due to interleave/portset behaviour, be processed by subscribers as

    Insert (item 1)
    Replace (with empty)
    Insert (item 2)

    leaving the subscriber in a different state to the publisher? This would be bad, and would not be what people would expect at a high-level from the Dss symmetric eventing model.

    Expecting that subscribers receive notifications in the correct order, even if this is done through some higher-level mechanism, would be a sufficiently common requirement from the programming model that this should be available in the DSS framework, rather than have each service roll-its-own.

     

    Wednesday, December 17, 2008 11:27 PM
  • Nick

    While I completely understand the point you are trying to make I think a better example is needed. (And I can't at present come up with one).

    I see 2 issues with the example:
    First the logic behind the publisher seems weird to me. Why would it be sending a replace (An updated state) at the beginning but not after each insert? (which could arrive out of order per your point) So in this case there should be 5 messages, 3 replaces and 2 inserts.

    And second, because of that I wonder why the subscriber would be subscribed to BOTH insert and replace as it seems to me that they get the same info from each. I admit though, the subscriber would need to compare both the lists in the state to see which was inserted if that were important. My point is I would expect a subscriber to need the info from only one source.

    Per your point directly:
    It would seem to me (the armchair QB) that order could be preserved not by ports in general but because as a message arrived the receiver task would be queued up to the interleaves associated task queue. Thus ordering across ports. If the tasks are queued up on message arrival then starvation should be dealt with but throughput might not. Problem is, I am just a spectator here and have no idea what is going on under the hood.

    Bryan

    Thursday, December 18, 2008 1:08 PM
  • Bryan hi,

    My understanding is that the publishing model is based on publishing the change that has just been applied. So if service A has collection state and service B subscribes it

    1. As part of the subscription set-up, A explicitly sends a 'fake' replace to B, i.e. A's state wasn't really replaced by anything, but Replace is the message that provides the full state, so that's what's sent to B, so B now has an 'initial' image.
    2. Some other service sends an Insert to A, adding an item. Assuming this succeeds, A publishes the insert via its subscription manager, which forwards the insert to B. B applies the insert however it sees fit.

    Logically, there isn't any need for A to publish a complete replace after an insert. And nor should there be, else for services with large collection state, performance would pretty much go down the pan.

    This symmetry is precisely why the portset in service B *is* the operations port of the service A.

    So I think the example holds (and the question remains): Service A is sent a replace (that clears its collection) followed by two inserts. Is service B *guaranteed* to process these notifications in order, given the whole interleave/portset configuration.

    Would someone from the robotics team care to comment - what does DSSP say about this, what should I expect from the programming model and what does the implementation actually guarantee?

    Thanks
    Nick

    Thursday, December 18, 2008 11:24 PM
  • Nick, you are correct that we will not preserve order across subscriptions, especially, across different message types, since they arehandled by different ports on the receiver end.
    Note however, that if the subscriber keeps track of the last message received timestamp, it can disregard any notification that was sent before that message (this indicates out of order). In the case of state modifying verbs, that rule can apply mostly to Replace: If you receive a replace, ignore any other notification (insert, update, Replace) that originated before it, its out of date.

    Our sopa envelopes include a timestamp, but its actually safer to either include a custome header (that is IDssSerializable) with a timestamp that is meaningful to your app, or include a DateTime field in the body.

    as you are well aware, while we can provide ordered notifications, we will do so at the expense of concurrency (since guaranteeing order across two independent entities means somehow introducing data dependencies between messages, manifested as a little send/receive/ack protocol)

    thanx
    g
    • Marked as answer by Trevor Taylor Wednesday, December 31, 2008 9:56 PM
    Tuesday, December 30, 2008 9:28 PM