none
Eric Lippert's Blog - CPS RRS feed

All replies

  • Cool, thanks.  (I hope the PDC vids will be re-playable. :)
    http://davesexton.com/blog
    Thursday, October 28, 2010 8:09 PM
  • There is a lot of new stuff in the today's release - do you guys have any materials explaining what is what? Maybe a video, as usual...
    Thursday, October 28, 2010 8:20 PM
  • Expect a couple of blog posts in the next few days. Videos will probably have to wait a bit as Charles is extremely busy with PDC..
    Thursday, October 28, 2010 8:56 PM
  • Hi everyone,

    Where's the love for Rx in the new async and await feature of C# 5?  How about some language support for something similar to Observable.Iterate?

    Does this even make sense?

    (The following hypothetical code was also posted as a comment on Eric's latest blog post.  It's based on Eric's own example of asynchrony using continuations and the aforementioned prototype C# 5 features.)

    async IObservable<Document> ArchiveDocumentsAsync(List<Url> urls)
    {
    	long count = 0;
    	Task archive = null;
    	Document previousDocument;
    
    	for(int i = 0; i < urls.Count; ++i)
    	{
    		var document = await FetchAsync(urls[i]);
    
    		count += document.Length;
    
    		if (archive != null)
    		{
    			await archive;
    			yield return previousDocument;
    		}
    
    		previousDocument = document;
    		archive = ArchiveAsync(document);
    	}
    }
    

    - Dave


    http://davesexton.com/blog
    Monday, November 1, 2010 2:53 PM
  • Could this be provided with a new static helper - Observable.CreateAsync<T>(Func<IObserver<T>, Task> ...)

    Where yield return is replaced with OnNext. Just a thought.


    James Miles http://enumeratethis.com
    Monday, November 1, 2010 3:17 PM
  • Hi, 

    I envision the compiler would produce something like the following (in terms of the observable; I'm not explicitly expanding upon the await feature and the generated state machine, of course.)

    async IObservable<Document> ArchiveDocumentsAsync(List<Url> urls)
    {
    	// EDIT: I forgot about the laziness of the IObservable pattern. The result
    	// of this method should be *cold*. Interestingly, since await and Observable.Create
    	// are both call/cc, the following line of code is just a quick representation of
    	// how Observable.Create would probably be used in the state machine that the C# compiler
    	// generates for the remainder of the method.
    	await Subscribe;
    
    	bool rethrow = false;
    	var observable = new Subject<Document>();
    	try
    	{
    		long count = 0;
    		Task archive = null;
    		Document previousDocument;
    
    		for (int i = 0; i < urls.Count; ++i)
    		{
    			var document = await FetchAsync(urls[i]);
    
    			// Naturally, "return" would actually be executed upon the first 
    			// call to await in the state machine that the C# compiler produces
    			// for this code.
    			// return observable.AsObservable();
    
    			count += document.Length;
    
    			if (archive != null)
    			{
    				await archive;
    				rethrow = true;
    				observable.OnNext(previousDocument);
    				rethrow = false;
    			}
    
    			previousDocument = document;
    			archive = ArchiveAsync(document);
    		}
    
    		rethrow = true;
    		observable.OnCompleted();
    	}
    	catch (Exception ex)
    	{
    		if (rethrow)
    			throw;
    		else
    			observable.OnError(ex);
    	}
    }

    - Dave


    http://davesexton.com/blog

    • Edited by Dave Sexton Monday, November 1, 2010 3:42 PM Added await Subscribe
    Monday, November 1, 2010 3:25 PM
  • Unfortunately only the GetAwaiter pattern is extensible, allowing different types to be 'awaited'. The pattern for returning from an async method is not extensible. For now we'd have to do something similar to the Observable.Iterate pattern, providing the observer as an argument. I'll discuss it with the team.


    Jeffrey

    Monday, November 1, 2010 4:53 PM
  • Hi Jeffrey,

    Thanks.

    Am I correct in seeing this as being a potential language-integrated replacement for Observable.Iterate?  I could even imagine FetchAsync returning IObservable<Document> instead of Task<Document>, allowing seemless flip-flopping between writing imperative-style observable sequences and using the monad with combinators.  For example:

    async IObservable<string> FirstLineOfEachArchivedDocumentAsync(IList<Url> urls)
    {
    	Document document = await ArchiveDocumentsAsync(urls).Where(d => d.LineCount > 0);
    	yield return document.FirstLine;
    }
    
    void PrintFirstLinesAndArchive(IList<Url> urls, int minLineLength)
    {
    	FirstLineOfEachArchivedDocumentAsync(urls)
    		.Where(line => line.Length >= minLineLength)
    		.Subscribe(Console.WriteLine);
    }
    

    Essentially, awaiting on ArchiveDocumentsAsync would be just like awaiting on Task<Document>, except that the current continuation would be invoked for each document that is observed from the IObservable<T> instead of the single document that would be produced by the Task<T>.

    - Dave


    http://davesexton.com/blog
    Monday, November 1, 2010 5:30 PM
  • Rx is about building streaming pipelines, though -- Task .. isn't.  Task is about the single result, or void result.  No multiple subscribers, either, which is an important feature of Rx.  I think it's best at this point not to conflate the two.

    Now, if one were able to create IObservable in the same way one creates IEnumerable... that would be nice love indeed.  But it's a different feature from await.

     


    Keith J. Farmer [Idea Entity]
    Monday, November 1, 2010 6:52 PM
  • Hi Keith,

    > Rx is about building streaming pipelines, though -- Task .. isn't.

    Well, Eric's blog posts seem to say otherwise about Task and await.  The whole thing is about Task and continuations, for designing complex control flows.  It's also about the ability to make await calls to other async methods, ultimately composing together one or more Task objects into an async pipeline.

    TDF is also about data flows over Task.

    > Task is about the single result, or void result.  No multiple subscribers, either, which is an important feature of Rx.

    Important, yet not required.  IObservable also supports a single subscription.  Furthermore, Task does support multiple subscriptions; namely, ContinueWith.

    I see IObservable as like Task but as a lazy sequence of values.  That's why they seem to fit so neatly together, IMO.

    > I think it's best at this point not to conflate the two.
    > Now, if one were able to create IObservable in the same way one creates IEnumerable...
    > that would be nice love indeed.  But it's a different feature from await.

    I think it's perfectly reasonable to conflate the two ideas into a single feature.  It would allow us to write declarative and composible asynchronous tasks as observable sequences, if we so choose.  In other words, we can choose to turn an asynchronous computation into a lazy sequence (IObservable), or we can convert it into a lazy result (Task).  It could be the developer's choice which is the most appropriate for their data model.

    Converting between Task and IObservable is already on the table, I believe.  So I don't see the problem here.

    Otherwise, without this language feature for IObservable, we're stuck inverting our minds with Observable.Iterate or we're forced to fit everything into the Task world, where problems related to sequences of data do not fit neatly.

    Think about it like this:

    To await anything is like adding a branch in the control flow.  You can await a task, or you can await the next value from an observable.  At that point, control is yielded to the caller and also to the task, simultaneously when necessary (that's up to the awaiter).  When you define an async method, hypothetically, you can specify that the result is a lazily-computed value (Task), or that it's a lazily-computed sequence of values (IObservable).  To compose an async pipeline is simply the act of awaiting on either of these to produce their value(s).  To yield a result is simply to push the next value to whomever is awaiting.

    - Dave

     


    http://davesexton.com/blog
    • Edited by Dave Sexton Monday, November 1, 2010 9:07 PM Changed "Interval" to "Iterate"
    Monday, November 1, 2010 8:09 PM
  • I tought Rx was actually about events and the composition of events.


    James Miles http://enumeratethis.com
    Monday, November 1, 2010 8:11 PM
  • Hi James,

    > I tought Rx was actually about events and the composition of events.

    What's the difference?  :)

    Essentially, IObservable<T> is a continuation monad.  IObserver<T> is the continuation.  Subscribe provides the action that executes when the observable generates a value.  Task represents some function or action.  It can also have one or more continuations.  ContinueWith provides the action that executes when the Task generates its value.

    The act of generating the value is the "event".  Rx does not require "events".  It simply requires notifications, which can be defined in many different forms; e.g., Notification<T>.

    What makes IObservable and Task async is the laziness that is achieved through their continuations.

    As for conflating the two with async and await (here's another thought related to Keith's reply), consider the following problem:

    async Task<IInfiniteSequence<T>> Forever()
    {
     // ?
    }
    

    Task does not model a sequence.  It's great for when you don't need a sequence.  Quite possibly, I'd recommend using Task over IObservable for methods that will only ever return a single value; perhaps it's easier to understand that way.

    In the example above, we have no choice but to use a Task with the prototype async and await feature.  That means our sequence of data will have to be entirely buffered before we can return to the caller.  In this case, it doesn't make sense because it's an infinite sequence, yet we have no way of modeling that now using async and await.

    IObservable does model a sequence.  Thus, theoretically, we could replace Task<IInfiniteSequence<T>> with IObservable<T> and get our results asynchronously, forever.  For consumers, usage would be similar:

    var task = Forever();
    task.ContinueWith(t => Use(t));
    task.ContinueWith(t => Abuse(t));
    
    var observable = Forever().Publish();
    observable.Subscribe(t => Use(t));
    observable.Subscribe(t => Abuse(t));
    observable.Connect();
    

    - Dave


    http://davesexton.com/blog
    Monday, November 1, 2010 9:04 PM
  • Hi Dave,

    I was being humerous ;)

    I completely agree, there is clearly a huge overlap here.


    James Miles http://enumeratethis.com

    Monday, November 1, 2010 10:09 PM
  • Hi James,

    That's good, because I did laugh ;)

    - Dave


    http://davesexton.com/blog
    Tuesday, November 2, 2010 8:14 AM
  • Great discussion! We agree that there is overlap between Task & IObservable. Both represent a continuation based approach to async programming. However, there are several key differences as some of you have already pointed out:

    1. IObservable is lazy(cold) and Task is strict(hot)
    2. IObservable represents a sequence of many values and Task contains at most one result.
    3. IObservable is an interface, Task is a concrete type
    4. IObservable has built in compositional cancellation, Task uses CancellationToken as an orthogonal mechanism.

    What does this mean?

    Because Task has at most 1 value, it lends itself extremely well to asynchronous sequential composition using structured control flow (await language support)

    Because IObservable is a push based sequence of values, it often requires merging streams simultaneously together. There is currently no way to express such operations using structured control flow. Therefore,  IObservable requires operators for parallel composition (LINQ). There is definitely more room to provide language syntax for IObservable, although less straight forward.

    Of course, as we have already seen, there are conversions between these two models.

    Which model should I use?

    If you're dealing with only asynchronous computations then Tasks and the await language support will provide a more familiar programming style.
    If you're dealing with a push sequences with a possible combination of asynchronous computations or if you desire a declarative programming style then use IObservable and the operators provided by Rx.

    Hope this helps,

    Wes & Jeffrey


    Thursday, November 4, 2010 9:11 PM
  • Hi Wes/Jeffrey,

    Thanks, that does help.  Seems like apt advice for a future version of the Rx Design Guidelines.

    Though your numbered list has me thinking (albeit probably nothing more than just a mental exercise): it seems that Task is merely a subset of IObservable.

    1. IObservable can be made hot; e.g., Publish.
    2. One is of course a subset of many.
    3. A concrete type is less flexible than an interface, in .NET.
    4. CancellationDisposable.

    In summary: laziness, sequences and polymorphism are a superset of strictness, singletons and concreteness.

    Therefore, couldn't all async methods that use Task function almost identically if they were to be implemented with IObservable instead?

    - Dave


    http://davesexton.com/blog
    Thursday, November 4, 2010 9:57 PM
  • Hi Dave,

     

    you're right in observing (pun intended) that IObservable can be made to do most of what Task does. The problem however is that it can do much more. As task deals with single values, the managed languages team considered it a better fit as it is easier mapped to existing language constructs as it follows sequential composition. Making it easier to implement the await feature as well as easier to understand by a broader audience. 

    We believe that IObservable and Rx is still the most complete solution for dealing with composition of any asynchronous computation and event stream. For those who need this kind of expressiveness or enjoy the declarative nature, we'll keep investing in Rx. We will also keep working with the managed language team to try to make IObservable fit in as well as possible in the patterns that their compilers generate. 

     

    Jeffrey

    Friday, November 5, 2010 3:13 AM
  • Hi Jeffrey,

    I should probably mention at this point that I like the new language feature.  :)

    I'm playing around with the CTP and I'm finding it to be pretty nice.  So I can understand why the language designers have made the decisions that they have.  Your extensions for IObservable and IAsyncEnumerable really help as well.

    > we'll keep investing in Rx.

    That's great to hear.  I did get the feeling, as did others it seems, that Rx is sort of being pushed (I can pun too!) aside by the new language feature and TDF.  One of my primary goals with these posts was to understand why IObservable didn't have a much bigger role in the new language feature.  I'm satisfied now with your answer to that question.

    Thanks, 
    Dave


    http://davesexton.com/blog
    Friday, November 5, 2010 5:11 AM
  • Hi,

      I'm butting in on a conversation that I don't fully understand, but I don't know if you caught the last 10 minutes of the PDC talk on C#.vNext - an update on compiler as a service was given. In particular, it DSL's were mentioned as one possible application they were targeting. After listening to the async part of the talk I had my hopes up that things like the async keyword could be writting using these extensions... This might mean you could better integrate some of Rx into the language by writing an extension. Of course, I would guess this will be a huge investment in work.

      Cheers, Gordon.


    Gordon
    Monday, November 15, 2010 6:38 AM
  • Hi Gordon,

    That would be great, but I believe they've been talking about "compiler as a service" for a while now.  I did hear it mentioned in the PDC, but my impression was that we won't be seeing it in C# vNext.  And anyway, integrating Rx-specific keywords and compiler behavior into C# myself kind of defeats the purpose.  For one thing, it's something that I don't have the knowledge or skills to even attempt :)

    - Dave


    http://davesexton.com/blog
    Monday, November 15, 2010 7:31 AM
  • Hi Dave,

      Ah, but that is the beuty. With a group of people you could figure out the right mix between keywords and library implementation. The power of LINQ, for example (and perhaps async, though I haven't looked at that carefully) is that it is a mix - so some general structure is put into the compiler, and the details are left to the library. Imagine if you could do what "IEnumerable<string> routine () { yield return "hi there" }" did for Observables? I would think that would be quite powerful.

      As far as how hard it is... I expect it will be the usual. First only really smart people will do it - and they will post a bunch of samples. Then people like us will look at those samples and edit a few things here and their and have something crude working, and then build on it. Sevearl years down the line, as we survey the *mess* we've made of C#, we'll be pissed at the compiler writers for making it so easy any nOob can write an extension! ;-)

      You are right, however. It isn't worth adding keywords when the library is simple - it is only when you find yourself having to write some nasty boilerplate code that you want to do something like modify the language.

      You are rigth about v.Next - it wasn't clear when that was going to happen - they have been working on it for a while...

      Cheers, Gordon.


    Gordon
    Tuesday, November 16, 2010 5:46 AM