none
It's hard to reason about Publish

All replies

  • Hi, 

    Try thinking about it like this:

    Defer makes a hot observable cold.
    Publish makes a cold observable hot.

    (In this respect, these two operators are dual.)

    So what are hot and cold observables?  If calling Subscribe on an observable sequence may cause side-effects, then you can say that the observable is cold; otherwise, it is hot.

    For example, Observable.Range returns a cold observable because each time that you call Subscribe you'll observe the entire range; i.e., the side-effect is that the range is recreated each time you call Subscribe.

    var o = Observable.Range(1, 10);
    
    o.Subscribe(Console.WriteLine); // prints 1-10
    o.Subscribe(Console.WriteLine); // prints 1-10
    

    If you were to Publish the observable first, then it becomes hot so that multiple calls to Subscribe share the same subscription side-effects, thus each subscription will observe the same continuous range.  This also means that you may call Subscribe on the hot observable after the Range has already begun pushing values, in which case late subscriptions simply won't observe any values they've missed.  That's why Publish returns an IConnectableObservable.  It allows you to call Subscribe multiple times to set up your queries first, and then when you're ready to share the subscription side-effects you'd call Connect once to subscribe to the source.

    Now consider that any .NET event can be made into a hot observable simply by calling FromEventPattern, without using Publish.  Take the following observable for instance:

    var o = Observable.FromEventPattern<MouseEventHandler, MouseEventArgs>(
    	eh => eh.Invoke, 
    	eh => MouseUp += eh, 
    	eh => MouseUp -= eh);
    

    This observable is hot because there are no side-effects when calling Subscribe - you'll simply observe mouse up events as they happen.  Furthermore, you may have already missed some events that were raised before you called Subscribe.  This is the nature of hot observables.

    Now, let's say that you want to make this hot observable cold so that subscription side-effects are not shared between multiple calls to Subscribe.  You can use Defer as follows:

    var d = Observable.Defer(() =>
    {
    	MessageBox.Show("Subscribed!");
    
    	return Observable.FromEventPattern<MouseEventHandler, MouseEventArgs>(
    		eh => eh.Invoke, 
    		eh => MouseUp += eh, 
    		eh => MouseUp -= eh);
    });
    

    Each time that d.Subscribe is called it shows a message box.  That's a subscription side-effect, which means that the observable is now cold.

    See this video for more information about hot and cold observables:

    http://channel9.msdn.com/Blogs/J.Van.Gogh/Rx-API-in-depth-Hot-and-Cold-observables

    - Dave


    http://davesexton.com/blog
    Thursday, July 14, 2011 5:43 PM
  • Hi Dave,

    I agree that `Publish` makes a cold observable hot, but I don't understand how `Defer` makes a hot observable cold.

    I thought the difference between hot and cold was summed up by:

    • Hot - if I wasn't subscribed when the event happened then I missed it.
    • Cold - no matter when I subscribe I'll get all the events.

    So `Publish` makes sense. If I publish a cold observable, connect to it, then subscribe after some events have happened then I miss those events - it's now hot.

    But `Defer` just seems to, well, allow hooking the subscription of an observable. Hot stays hot and cold stays cold - at least under my understanding.

    Can you help clear up my confusion on this now?

    Cheers.


    James C-S
    Friday, July 15, 2011 1:48 AM
  • Hi James,

    By your definition, Replay is cold because when you subscribe you'll get all of the notifications.  But we know that internally Replay inserts a ReplaySubject and it returns IConnectableObservable, because it's actually hot.

    Now imagine a cold observable with the following behavior: The first subscription gets all of the values and all subsequent subscriptions get none.  This would require a side-effect, also implying statefulness; e.g., perhaps it involves flipping a boolean switch upon the first call to Subscribe.  By your definition, this observable is hot because there's a race condition that causes the late subscriptions to miss values; however, it's actually cold because the race condition is imposed by a side-effect.

    It's whether an observable causes subscription side-effects that makes it hot or cold.

    The fact that generally hot observables are already "running" and cold observables are always replayed is due to the convenience of these behaviors.  Here's an interesting fact about an observable created by FromEventPattern:  When writing a custom .NET event you can define an add method to manually write the code that adds event handlers to the underlying delegate.  This code can also do anything else that a typical .NET method can do.  In other words, we can introduce subscription side-effects even for the observable returned by FromEventPattern.  So technically, even though we think of observables returned by FromEventPattern as hot, they can be cold!  Typically side-effects do not occur when registering events, so it's generally safe to assume that they are hot, although the C# language let's us add subscription side-effects anywhere that we want, blurring the line a bit between hot and cold sequences.

    Let's assume that C# didn't provide any mechanism to cause side-effects when subscribing to an observable returned by FromEventPattern.  How could you add some subscription side-effects then?  Defer allows you to add subscription side-effects to a hot observable, thus it can make a hot observable cold.

    --

    Note that I thought the same way as you once :)   Check out this old post of mine, including Wes's response:

    http://social.msdn.microsoft.com/Forums/en-US/rx/thread/8899f426-ea08-43d5-869a-d1d60e394d23

    He also confirms in his final reply that Defer and Publish are dual.

    My reply to Wes's answer was a bit short-sided.   I can see that now, but at the time I was a bit confused by the meaning of "broadcasting" and "subscription side-effects", and how they overlap.  It became clearer to me over time as I realized how important it is to consider subscription side-effects.  The fact that Publish broadcasts is merely how it shares subscriptions side-effects; essentially, these properties of Publish are one and the same.

    - Dave


    http://davesexton.com/blog
    Friday, July 15, 2011 3:03 AM
  • Hi James,

    Let's also apply our definitions to enumerables:

    An IEnumerable<T> that is returned by an iterator block is cold.
    An IEnumerable<T> that is cast from an IList<T> is hot

    These definitions agree with my definition:

    > It's whether an observable causes subscription side-effects that makes it hot or cold.

    An iterator block is cold because iteration can cause side-effects.
    IList<T> is hot because iteration cannot cause side-effects.

    However, these definitions disagree with your definitions:

    > Hot - if I wasn't subscribed when the event happened then I missed it.
    > Cold - no matter when I subscribe I'll get all the events.

    In the interactive world, "miss" implies that the sequence can change between new iterations, which means that side-effects can occur.  Therefore, by your definition a hot enumerable can have side-effects, so an iterator block is hot.

    In the interactive world, "getting all of the events" can be interpreted to mean that the underlying sequence is immutable, which implies that side-effects cannot occur.  Therefore, by your definition a cold enumerable cannot have side-effects, so an IList<T> is cold.

    Your definitions are exactly opposite in the interactive world!

    - Dave


    http://davesexton.com/blog

    • Edited by Dave Sexton Friday, July 15, 2011 3:23 AM Some wording and formatting updates
    Friday, July 15, 2011 3:21 AM
  • Hi Dave,

    I'm not convinced yet. I went and watched Wes' video on Channel 9 - http://channel9.msdn.com/Blogs/J.Van.Gogh/Rx-API-in-depth-Hot-and-Cold-observables - and grabbed these quotes:

    Wes says, "A hot observable is one that kind of exists whether or not you're subscribed to it. So these things are streaming in whether or not I'm subscribed to them."

    And on cold observables, he says, "With this guy it doesn't actually run the code that creates the message OnNext until the point which I subscribe and it happens each time I subscribe as well."

    So it sounds a little like your definition of subscription side-effects, but if I take Wes' example observables and examine their behaviours and the behaviours when they are deferred I get a different definition. Especially when I look in Reflector I get the opposite subscription side-effects.

    Wes' examples in the video of hot and cold observables are:

    var hot = Observable.FromEventPattern(/* MouseMoves */);
    var cold = Observable.Return(5);
    

    If we defer them, we get:

    var deferredHot = Observable.Defer(() => hot);
    var deferredCold = Observable.Defer(() => cold);
    

    Using Reflector it can be seen that:

    `hot` is an `AnonymousObservable` that, for every new observer subscription, adds a handler to an event and calls `OnNext` for every subsequently raised event.
    `cold` is an `AnonymousObservable` that, for every new observer subscription, calls `OnNext` immediately with the value 5.

    `deferredHot` is an `AnonymousObservable` that, for every new observer subscription, returns the underlying `AnonymousObservable` that adds a handler to an event and calls `OnNext` for every subsequently raised event.
    `deferredCold` is an `AnonymousObservable` that, for every new observer subscription, returns the underlying `AnonymousObservable` that calls `OnNext` immediately with the value 5.

    Therefore the semantics for `hot` & `deferredHot` are the same, and the semantics for `cold` & `deferredCold` are the same.

    Also `hot` causes a side-effect by subscribing to the event handler (which it does for every subscription) and `cold` does not (in this case) cause any side-effects.

    I agree that cold observables "can" cause side-effects, but don't have to, and hot observables certainly can cause side-effects (ie add handler in this case).

    More importantly `Defer` doesn't change the semantics of either an existing hot or cold observable so it can't be said that `Defer` makes hot observables cold.

    It seems to me that my original definition for hot observables is correct. My cold observable definition is probably better written as "no matter when I subscribe I'll get all the events *computed at the time of the subscription*."

    Dave, I know you know your stuff, so I'm probably missing something, so please help me out.

    Cheers.
    James C-S
    Friday, July 15, 2011 9:34 AM
  • Hi James,

    Your points are good, so it's hard to disagree, but I think there are still some subtleties that perhaps you're overlooking.  This has always been a confusing topic for me because there are a lot of factors to consider, but I'm glad to finally discuss it in depth.  Perhaps we'll get to the root of the confusion :)

    > Wes says, "A hot observable is one that kind of exists whether or not you're subscribed to it. So these things
    > are streaming in whether or not I'm subscribed to them."

    I believe that's just the typical behavior of hot observables though, so it's convenient to think of them this way.  But as I've already pointed out, the line can be blurred.

    Consider an observable that has subscription side-effects and grabs its value from a hot source, such as time; e.g., DateTime.Now.  Is this observable hot?  In a sense it is, because we tend to think of hot observables as "already running" and time certainly fits that criteria.  But let's not overlook the fact that each time you subscribe to this observable, code is executed that causes side-effects.

    If somebody were to ask you about this observable, would you tell them that it's hot even though it has subscription side-effects?  That would be misleading, because generally we think of hot observables as not having any subscription side-effects.  So would you tell them that it's cold?  This is much safer, because it will make them think twice about writing a query like xs.Zip(xs, (l,r) => l + r) without using Publish to prevent duplicate side-effects.

    A sequence that takes values from a hot source isn't necessarily hot itself.

    Wes' examples in the video of hot and cold observables are:
    var hot = Observable.FromEventPattern(/* MouseMoves */);
    var cold = Observable.Return(5);
    

    If we defer them, we get:
    var deferredHot = Observable.Defer(() => hot);
    var deferredCold = Observable.Defer(() => cold);
    

    C# lets us do strange things. As I pointed out before, FromEventPattern can execute code that we've written in an add method, which could of course have side-effects.  If FromEventPattern calls a side-effecting add method, does it still produce a hot observable?  We want to say yes, because we think of events as being hot, but just because the source is hot doesn't mean that the new sequence is hot.  If we make the (probably dumb) decision to cause side-effects each time that a handler is added, then we must refer to the new sequence as being cold so that people are aware of this strange behavior.  Even though it's a sequence of event notifications, a consumer might want to use Publish to prevent duplicating subscription side-effects.

    > Using Reflector it can be seen that:
    > [snip]
    > Therefore the semantics for `hot` & `deferredHot` are the same, and the semantics for `cold` & `deferredCold` are the same.

    Agreed.  But there's one problem here: the examples are unrealistic.  What's the point of deffering a cold observable if you're not going to add additional side-effects?  Likewise, what's the point of deferring a hot observable if you're not causing side-effects?

    So it seems that it's not the potential for adding side-effects that makes Defer turn a hot observable into a cold observable.  It's the actual side-effects that changes an observable's temperature.  And for a hot observable, what better way is there to add subscription side-effects than Defer?

    > Also `hot` causes a side-effect by subscribing to the event handler (which it does for every subscription) and `cold` does
    > not (in this case) cause any side-effects. [snip]

    Is that really a side-effect though, or is that just the primary effect?  Adding an event handler is merely how we subscribe to events.  Adding an observer is how we subscribe to observables.  Perhaps if C# had first-class support for subscribing to events as observables, we would ignore the effect of event registration and just assume that it happens magically behind the scenes ;)

    > More importantly `Defer` doesn't change the semantics of either an existing hot or cold observable so it can't be
    > said that `Defer` makes hot observables cold.

    Whether an observable generates values from a hot source or not is part of the observable's semantics, whereas the state of being hot or cold is just a technicality that indicates how we should consider side-effects when using a sequence in a query.

    It's probably easier to think in terms of operators when considering semantics vs. technicality.  For example, I can write an operator called FromEventPattern that returns an observable of event notifications.  The fact that the underlying data comes from a hot source is part of the operator's semantics.  I can also write an operator called FromEventPatternAndFormatCDrive.  The fact that it formats the C drive is also part of the operator's semantics.  But does it format the C drive every time that I subscribe to the observable that it returns?  Or does it format the C drive only when the operator is called?

    Often the semantics of an operator implies whether it's hot or cold, so for convenience we tend to eliminate the difference.  We assume that FromEventPattern is hot because it wraps a hot source and generally it's a safe assumption that listening for events doesn't cause any side-effects.  We assume that Defer is cold, because it's safer to assume that subscribing causes side-effects, just in case it does.  Therefore, we must assume that FromEventPatternAndFormatCDrive is cold just in case it is, even though it might not be.

    - Dave


    http://davesexton.com/blog
    Friday, July 15, 2011 6:37 PM