none
LinkTo predicates

    General discussion

  • i was doodling some datflow networks and ran into an issue with LinkTo, predicates and the discardsMessages boolean. I had some links with the last one discardsMessages set to true, but then i added another link to another block, that block would never get any messages since the previous link was discarding them. As far as i understand there is no way to change the discardsMessages value.

    im not sure how i feel about having that option in the first place.. i can see the need for it, but i can also feel like it doesnt belong on the link, but rather on the source block. i suppose the question is what it is you want to have happen if nobody wants a perticular message. BufferBlock waits forever until someone who wants the message comes along, Broadcast waits until another message comes in. In both cases though, its the source that decides.

    i kind of wish there was a SwitchBlock to witch you could add predicates and associated targetblocks, but that also had a 'Default' source that it could post messages to that didnt match any predicate.

    Another, perhaps cleaner solution would be if there where If blocks that takes a T and then expose two sources also of T, one if the supplied predicate is true and another if its false. That block could have an option to decline message that doesnt match the predicate, or post them to its own False source [that could be linked  wherever]

    if im understaning correctly, LinkTo already uses a If block of sorts, its just not public.

    Wednesday, February 09, 2011 10:48 PM

All replies

  • perhaps something like this:

     

    public class IfBlock<T> : ITargetBlock<T> {
      private Func<T, bool> _predicate = null;
      public ITargetBlock<T> True { get; set; }
      public ITargetBlock<T> False { get; set; }
      public bool DeclineOnFalse { get; set; }
      public void DeclinePermanently( ) { throw new NotImplementedException( ); }
      public Task CompletionTask { get { throw new NotImplementedException( ); } }
    
      public IfBlock( Func<T, bool> predicate ) {
       _predicate = predicate;
       True = new BufferBlock<T>( );
       False = new BufferBlock<T>( );
      }
    
      public DataflowMessageStatus OfferMessage( DataflowMessage<T> message, ISourceBlock<T> source, bool consumeToAccept ) {
       var match = _predicate( message.Value );
       if( match ) {
        return True.OfferMessage( message, source, consumeToAccept );
       }
       else {
        if( DeclineOnFalse )
         return DataflowMessageStatus.Declined;
        else {
         return False.OfferMessage( message, source, consumeToAccept );
        }
       }
      }
    
      public bool Post( T item ) {
       if( _predicate( item ) )
        return True.Post( item );
       else if( DeclineOnFalse )
        return false;
       else
        return False.Post( item );
      }
     }
    

     

    Wednesday, February 09, 2011 11:20 PM
  • Hi Allan,

     

    discardMessages on LinkTo is no worse than linking a block that consumes all offered messages. Take this example for instance:

     

    var source = new BufferBlock<int>();

    var sweeper = new BufferBlock<int>(); // this block accepts all offered messages

    var doomed = new ActionBlock<int>(x => ConsoleWriteLine(“A miracle has happened!”)); // This block will never get a message when linked after sweeper

     

    var sweeperLink = source.LinkTo(sweeper);

    sourceLinkTo(doomed);

     

    Console.WriteLine(“Do you believe in miracles?”);

    for (int i = 0; i < 1000000000; i++) source.Post(i);

    Console.WriteLine(“I guess not.”);

     

    Console.WriteLine(“Alright. Enjoy!”);

    sweeperLink.Dispose();

    source.Post(1);

     

    The discardMessages option is not used, but the doomed block still doesn’t get any message as long as the sweeper remains linked, because the sweeper accepts everything. So don’t take it too hard on discardMessages – this is how the network was constructed. The moment the sweeping link is disposed of, a miracle will happen.

     

     

    With your “If” block, you are getting to the rationale behind LinkTo. LinkTo uses an internal “conditional synchronous propagator” block. That allows you to easily model what you want:

    a) forward all verified messages to verifiedTtarget, and decline all unverified messages:

    source.LinkTo(verifiedTarget, verification);

     

    b) forward all verified messages to verifiedTtarget, and forward all unverified messages to unverifiedTarget:

    source.LinkTo(verifiedTarget, verification);

    source.LinkTo(unverifiedTarget);

     

    c) forward all verified messages to verifiedTtarget, and discard all unverified messages:

    source.LinkTo(verifiedTarget, verification, discardMessages: true);

     

    As you can see you can easily model “if” forking as well as “switch” forking. Our goal is to give you a set of primitives that enable you to build the network that will solve your business problem.

     

     

    Zlatko Michailov

    Software Development Engineer, Parallel Computing Platform

    Microsoft Corp.

     

    If you are satisfied by this post, please mark it as “Answer”.

     


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, February 10, 2011 7:58 PM
  • it is worse because you cant change or even access the discardMessages once the link has been established.. if i have an empty actionblock that effectively sinks unhandled messages, i can atleast dispose that link later without affecting the predicated block. yes, you could do that right now as well, but i think it would be cleaner to not have the discardMessages option at all.

    i understand that you can create if statements using the LinkTo predicate, im just opposed to how it works. it hides away very significant information about the network. that information doesnt belong on the link, it should be its own block.. its also annoying that i have to specify [and run] the predicate twice if i want to do an If/else, one for the 'true' case and one for the 'false'.

    i think the api would be much clearer and composable if the predicate had its own block. after all that is how it is done internally already. it is also how wf does it

    Thursday, February 10, 2011 8:58 PM
  • If I understand your first point correctly, you are suggesting removing the discardMessage option on LinkTo, because it could be simulated by linking a dummy ActionBlock. Logically, that is true. Practically, that ActionBlock will be consuming memory to buffer up messages before being able to “vaporize” them. Then it will spin up tasks that will consume CPU cycles to get those tasks scheduled, to pop buffered messages, and to invoke the dummy callback. I’m pretty sure some people will gladly opt into using discardMessages to avoid that cost.

     

    Re: “its also annoying that i have to specify [and run] the predicate twice if i want to do an If/else, one for the 'true' case and one for the 'false'

    Unless the source is a BroadcastBlock, the predicate need not be invoked more than once per message. See the examples I listed above. And BroadcastBlock doesn’t make sense to be used as a source in an “if/else” fork because its sole purpose is to make message acceptance non-exclusive.

     

    Re: “the api would be much clearer and composable if the predicate had its own block. after all that is how it is done internally already

    We've been discussing this. Your opinion is well taken. That doesn’t mean we are considering removing the predicate on LinkTo though.

     

     

    Zlatko Michailov

    Software Development Engineer, Parallel Computing Platform

    Microsoft Corp.

     

    If you are satisfied by this post, please mark it as “Answer”.

     


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, February 10, 2011 10:20 PM
  • " I’m pretty sure some people will gladly opt into using discardMessages to avoid that cost"

    i agree -if- you use an ActionBlock :) but you could have a SinkBlock or NullBlock or something that just throws messages away in a synchronous fashion

    "Unless the source is a BroadcastBlock, the predicate need not be invoked more than once per message"

    i see now that you are correct. for some reason i had it that you'd have to do a.LinkTo(t => t.checkIfTrue) and also a.LinkTo(t => t.checkIfFalse) but that is obviously not the case..

    still, i'd really like an If block :) i think it whould make networks easier to follow, and also help toolability in the future.

    Friday, February 11, 2011 8:53 AM
  • Thanks for the great input, Allan (please keep it coming!).  We'll definitely consider it.
    Friday, February 11, 2011 3:35 PM