Observable over the wire

Question

I'm looking to provide push-based notifications to clients over the wire and using and the IObservable/IObserver interfaces are obviously a great fit.

I was going to implement WS-Eventing (to ensure the web interface follows a standard) which could then be easily wrapped to look like an IObserver/IObservable on the WCF client/server.

However, with Rx now being shipped under the "Microsoft Cloud Programmability" flag, I can only assume one of the first TODO's for the team is to provide a networked implementation of IObservable pattern.

I also found a post were Jeffrey mentioned a planned WCF proxy class generator.

http://social.msdn.microsoft.com/Forums/en/rx/thread/cf98ee15-abbd-4317-86e9-228fda4bab3e

Would anyone on the Rx team care to update the community on their intentions wrt a networked Observable implementation?

I'd prefer not write my own only to see you provide the same (most likely better :) funcitonality a week later?

Thanks!

Answer 1

Hi,

What about IQbservable?

http://channel9.msdn.com/shows/Going+Deep/Bart-De-Smet-Observations-on-IQbservable-The-Dual-of-IQueryable/

- Dave

---

http://davesexton.com/blog

Answer 2

Hmmm...not really sure how IQbservable could help me as I'm looking to send the actual data over the wire.

I've basically got a producer (IObservable) on one box, and a consumer (IObserver) on another. When my consumer finds out about a producer, it would subscribe and the producer would send notifications (OnNext, OnError, OnComplete) over a network connection (yes this implies a Queue on the producer if I want all notifications to arrive at the consumer).

Thanks!

Answer 3

Hi,

IQbservable would allow you to create an observable abstraction over your normal service calls, but I guess that's not what you're asking :)

Are you instead thinking about a model like the following?

[ Client (IObserver) ]                              [ Server (IObservable) ]

                   o ------------ Get IObservable ----------->

 IObservable <-------------- IObservable --------------o

                   o--------- Subscribe(IObserver) --------> IObserver

  IDisposable <--------------- IDisposable ---------------o

                  <---------- IObserver.OnNext(T) ---------o

                  <------ IObserver.OnCompleted(T) ------o

Essentially, you'd need bidirectional services.  I'm not sure if you'll be able to do that without remoting (or WCF).  I guess you could set it up so that you expose two normal services, and the method that gets the IObservable could be collapsed into a single call with additional info about the response service; e.g.:

[OperationContract]
Token Subscribe(IObserverServiceInfo observer);

[OperationContract]
void Cancel(Token);

And of course, if the service is actually using an IObservable internally then you could wrap the IObserverServiceInfo with a custom IObserver implementation to marshal calls to the other box, then subscribe normally.

Edit: I guess you'd have to do something similar for IDisposable as well, although the server information will already be present on the client so you'd need some helper method that simply wraps the result and calls another service for cancellation; i.e., you probably wouldn't be able to return IDisposable from the service.

- Dave

---

http://davesexton.com/blog

Answer 4

Yes thanks Dave for your reply. I know I could abstract the Observer pattern and implement the calls over WCF. In my original post I suggested I may even do so following the WS-Eventing spec which defines all of the required request/reply semantics which would allow my non-.Net clients the ability to subscribe to these "event streams" using a well known standard.

However, I don't want to do so if the Rx team is already doing something similar!

I know the team is probably busy with the next Rx version, but if anyone has any idea whether something like this is in the pipe I'd really appreciate it as I'd prefer not duplicating their efforts.

Thanks!

Answer 5

Hi,

I just looked up WS-Eventing and now I see what you mean - there is some parity there with IObservable and IObserver; it's plainly evident in the terminology used in the standard.

But the point of my last post was more about showing how this pattern doesn't really fit into IObservable nicely.  Of course, that doesn't mean it's not possible to create another abstraction over IObservable and it certainly doesn't mean that the Rx team isn't working on it (I'm curious now too :) but it seems to me that to have a working solution will probably require some significant functionality on top of the existing contracts of IObservable and IObserver to allow for things like proper error handling (e.g., what does the observable service do when a subscriber service throws an exception?), network (un)reliability (e.g., Does a failed connection warrant automatic renewal pending an immediate call to OnError?), notification synchronization (e.g., should OnNext calls block since they do now for local subscriptions?), and probably lots of other stuff I'm not thinking about.

Again, I'm not saying it can't be done or that it shouldn't be done, but it does seem like currently the IObservable and IObserver contracts aren't designed for distributed programming (ironically?).

- Dave

---

http://davesexton.com/blog

Answer 6

> but it does seem like currently the IObservable andIObserver contracts
> aren't designed for distributed programming (ironically?).

That would indeed be ironic since Rx originated from the learning of a previous project on distributed programming (http://www.infoq.com/articles/qcon-2007-bloggers-summary) and basically formalizes the well-know subject/observer design pattern (http://en.wikipedia.org/wiki/Observer_pattern); I think it works pretty well for composing WCF-based webservices (http://headinthebox.posterous.com/rx-sample). 

However, note that the word "event" is one of the most ambiguous and overload terms in CS, so it might be the case that the notion of "event" from Rx does not match with the notion of "event" in WS-eventing. Which by the way is why we did not pick the name LINQ to Events for Rx.

I am very much interested in SFJSI's experience of implementing WS-eventing using/on top of Rx. 

Answer 7

Hi Erik,

I see, so perhaps "distributed programming" wasn't the correct term for me to use :)

But still, I think I brought up reasonable questions about the differences between the contracts for these interfaces as they are used for local computing vs. how they could be used for distributed computing.  It seems in the local world the contracts may be more strict than what could be relied upon over a network.

The sample on your blog keeps both interfaces on the local side, so the parity is there.  IObservable is used locally merely as a facade for normal service calls.  But I don't think this meets the requirements of the poster.

- Dave

---

http://davesexton.com/blog

Answer 8

Agreed, that why "I am very much interested in SFJSI's experience of implementing WS-eventing using/on top of Rx. ".

Answer 9

Hi Erik,

I'm happy to have piqued your interest. I agree the notion of events in WS-Eventing doesn't exactly fit with Rx. The standard also has tons of non-observable functionality I don't want/care to think about (things like expiration, status, re-subscription etc). 

I'd prefer a "pure Observable" implementation using WCF contracts/endpoints. When interop is a requirement, such an implementation COULD be made to look like WS-Eventing (or some other pub/sub standard). As Dave wrote, there are many questions to be answered for all the possible edge/error cases but I'm not all that worried about those. There is really no technical barriers I can see in implementing such a thing.

I did ask the question here though just in case the Rx team was planning some sort of WCF-based reactive component. And to me it seems they would have to be thinking about this in order to provide a cloud-ready/distributed environment for reactive programming...

Thanks!

Shawn

Answer 10

I think this would be a most excellent community contribution. We want to build the bricks using which customers like you can build their homes.