none
Create TouchClick from TouchDown and TouchUp

    Question

  • I have learned a lot from the answers to my previous question re 'Long touch' (http://social.msdn.microsoft.com/Forums/en-US/92c26fe7-ee36-42fd-8eaf-bc938c219125/create-holding-event-from-mousedown-that-is-not-followed-by-mouseup-within-x-ms?forum=rx) but still not enough to successfully do something else: generate a 'Click' event from TouchDown followed by TouchUp events. The condition, which I forgot to include in my previous question is that the distance between the TouchUp and TouchDown points should not exceed some value (say 9 pixels). I tried the following but that blocked my UI. Fortunately, the compiler included a message that my First and Last are blocking and I should use their Async versions. I then got stuck on an error in that Where does not like async delegates, which I would need to use an Async function. Any help would be greatly appreciated.

    touches = 
        touchDowns.Window(() => touchUps)
        .Where(downToUps =>
            {
                var p1 = downToUps.First().EventArgs.GetTouchPoint(_container); // touch down point
                var p2 = downToUps.Last().EventArgs.GetTouchPoint(_container);  // touch lift (up) point
                return (Math.Pow(p1.Position.X - p2.Position.X, 2) + Math.Pow(p1.Position.Y - p2.Position.Y, 2) < 81);
            })
        .Select(downToUps => downToUps.Last().EventArgs.GetTouchPoint(_container).Position);
    There is an additional requirement I omitted, which I am not sure how to implement either: touchUps used in the Window expression may only be those whose device id (EventArgs.TouchDevice.Id) is the same as the touchDowns that are being windowed. That is touch down/up event pair must be caused by the same device id to be a pair.


    Sunday, September 28, 2014 8:47 AM

Answers

All replies

  • Hi,

    > touchUps used in the Window expression may only be those whose device id (EventArgs.TouchDevice.Id) is the same as the touchDowns

    To address your second requirement first, you can use grouping.

    For example: (Untested)

    touchDowns.GroupBy(e => e.DeviceId)
              .Select(g => g.Window(() => touchUps.Where(e => e.DeviceId == g.Key)))
              .Merge();

    > the distance between the TouchUp and TouchDown points should not exceed some value (say 9 pixels).

    This requirement can be met by using an overload of Window that makes the window's opening value available to the closing selector function.

    Technically, Window is just a specialization of the GroupJoin operator. This overload in particular is really just API sugar for GroupJoin.

    Here's a nice video intro to joining by coincidence in Rx.

    For example: (Untested)

    touchDowns.GroupBy(e => e.DeviceId)
              .Select(g => g.Window(g, 
                 down => touchUps.Where(up => up.DeviceId == g.Key && IsWithinRange(down, up))))
              .Merge();

    - Dave


    http://davesexton.com/blog

    Sunday, September 28, 2014 7:25 PM
  • After rereading your question, I see that I've probably misunderstood your distance requirement. You don't want to ignore up events exceeding the distance in the Window operator because that will prevent some windows from closing.  You simply want to filter away notifications in the outer sequence that were caused by a pair of events father apart than a certain distance. Correct?

    For that I'd probably use GroupJoin instead of Window because it allows you to project your own data. In your case, you could project an anonymous type containing the down and up notifications, then simply add a Where clause at the end of the query, and a Select in case you want to drop the up notifications from T.

    Try it yourself. If you want to cheat a bit, you can look at the source code for the Window operator that I mentioned in my previous reply to see how it calls GroupJoin.

    - Dave


    http://davesexton.com/blog

    Sunday, September 28, 2014 7:36 PM
  • Yes, you are correct, I want to ignore any pairs of events, which occurred beyond certain distance from each other. I am having trouble though understanding from the video or blogs how to define the duration functions GroupJoin expects. To simplify my test environment I went back to using mouse events, i.e. no requirement for verifying device id and for now even distance between the down and up event. I constructed the following sequence. However, it gives me a tuple with just the Down event and empty Ups component as soon as I press the mouse down and does not give any event when the mouse goes up. It seems like my definition function is incorrect but I have no idea how to fix it.

    mouseDowns = Observable.FromEventPattern<MouseButtonEventArgs>(_container, "MouseDown");
    mouseUps = Observable.FromEventPattern<MouseButtonEventArgs>(_container, "MouseUp");
    var clicks = mouseDowns.GroupJoin(
                    mouseUps,
                    _ => Observable.Never<Unit>(),
                    _ => Observable.Empty<Unit>(),
                    (downs, ups) => Tuple.Create(downs, ups));
    clicks.Subscribe(t =>
                    {
                        Debug.WriteLine("Down: " + t.Item1.EventArgs.LeftButton);
                        t.Item2.Do((i2) => Debug.WriteLine("Up: " + i2.EventArgs.LeftButton));
                    });

    Monday, September 29, 2014 12:04 AM
  • You're on the right track. You just forgot to subscribe to the Do query. (Edit: Probably should just call Subscribe instead of Do.)

    You'll probably want to close your down windows too, which means also passing in mouseUps as the closer rather than Never.

    Next, instead of the Tuple try a SelectMany query.  You can do your distance comparison with the original down event still in scope with each up event (though you know there will only be one up event per down event, SelectMany doesn't have to know that).

    - Dave


    http://davesexton.com/blog

    • Edited by Dave Sexton Monday, September 29, 2014 12:16 AM
    Monday, September 29, 2014 12:15 AM
  • Here's the pretty version of the query:

    (Edit: Actually there's a subtle bug in this query. Can you spot it? Hint: It's not a bug with the query itself, per se)

    var clicks = from down in downs
                 join up in ups
                 on ups equals Observable.Empty<Unit>()
                 into window
                 from up in window
                 where up.GetPosition(this).X <= down.GetPosition(this).X + 50
                 select new { down, up };

    - Dave

    http://davesexton.com/blog

    • Edited by Dave Sexton Monday, September 29, 2014 12:25 AM
    Monday, September 29, 2014 12:23 AM
  • Thanks for all your help. I really appreciate it.

    I fixed the Do mistake. However, the problem is that the (first) Subscribe gets called as soon as I press the mouse down, rather than when the mouse goes up.

    I don't understand your statement: "You'll probably want to close your down windows too, which means also passing in mouseUps as the closer rather than Never." How are these delegates being used to indicate duration? The GroupJoin signature includes left and right 'duration selector' functions. How do they act as selectors? What do they select?

    Monday, September 29, 2014 12:42 AM
  • Try running my pretty query and let me know if it works as you'd expect (step 1).  Though as noted, there's a subtle bug - not with the query itself, but with the data. I'm sure you'll spot the problem after you see the output.

    > I don't understand your statement [snip]

    I meant for you to do as I did in my example query.  I'm joining to ups and I'm using ups again as the closer for downs.  It appears in my query twice.  Without a closer for downs, every up event will be projected into every down event because the window created for each down event never ends!


    http://davesexton.com/blog

    • Edited by Dave Sexton Monday, September 29, 2014 12:50 AM
    Monday, September 29, 2014 12:50 AM
  • Thanks for the challenge! I was getting bored :)

    A subtle bug in your example was that GetPosition returns the current position, not the position at the time the original mouse event was raised. Consequently, the 'where' clause was always true no matter how much the mouse had moved from down to up position.

    I modified the example to handle multi-touch devices. However, my modification (the ups.Where clause below) is counter-intuitive. I don't understand what am I comparing to what through the equals operator. The code seems to work OK - I just don't like code by trial and error (and a lot of help from the community).

    touchDowns = Observable.FromEventPattern<TouchEventArgs>(container, "TouchDown");
    touchUps = Observable.FromEventPattern<TouchEventArgs>(container, "TouchUp");
    var downs = from down in touchDowns select new { Pos = down.EventArgs.GetTouchPoint(_container), Id = down.EventArgs.TouchDevice.Id };
    var ups = from up in touchUps select new { Pos = up.EventArgs.GetTouchPoint(_container), Id = up.EventArgs.TouchDevice.Id };
    touches = from down in downs
                    join up in ups on ups.Where(u => u.Id == down.Id) equals Observable.Empty<Unit>()
                    into window
                    from up in window
                    where Math.Pow(up.Pos.Position.X - down.Pos.Position.X, 2) + Math.Pow(up.Pos.Position.Y - down.Pos.Position.Y, 2) <= 81.0
                    select down.Pos;

    Monday, September 29, 2014 4:11 AM
  • > A subtle bug in your example was that GetPosition returns the current position, not the position at the time the original mouse event was raised.

    Yep, that's the bug. It's funny because I do this every time. It's easy to forget that GetPosition must be projected immediately.

    Your query looks great. It's nice that you didn't even need GroupBy. The GroupJoin operator is quite powerful indeed.

    Note that another purpose of using GroupJoin instead of Join was so that you could throttle your query (as per our previous discussion):

    from up in window.StartWith(down).Throttle(...)

    However, if you don't need to throttle, then you could simplify your query into a regular Join, eliminating the window (a.k.a., group) entirely as shown in the following example.

    I'd also like to point out again that GroupJoin is the primitive operator for joining by coincidence. The Join operator (and of course Window) can be implemented in terms of GroupJoin, as shown by the fact that we can reduce your query by simply eliminating window.

    touchDowns = Observable.FromEventPattern<TouchEventArgs>(container, "TouchDown");
    touchUps = Observable.FromEventPattern<TouchEventArgs>(container, "TouchUp");
    var downs = from down in touchDowns select new { Pos = down.EventArgs.GetTouchPoint(_container), Id = down.EventArgs.TouchDevice.Id };
    var ups = from up in touchUps select new { Pos = up.EventArgs.GetTouchPoint(_container), Id = up.EventArgs.TouchDevice.Id };
    touches = from down in downs
                    join up in ups on ups.Where(u => u.Id == down.Id) equals Observable.Empty<Unit>()
                    where Math.Pow(up.Pos.Position.X - down.Pos.Position.X, 2) + Math.Pow(up.Pos.Position.Y - down.Pos.Position.Y, 2) <= 81.0
                    select down.Pos;

    - Dave


    http://davesexton.com/blog

    Monday, September 29, 2014 9:21 AM
  • > I don't understand what am I comparing to what through the equals operator

    You're simply comparing durations. Joining by coincidence means joining events that occur within overlapping durations.

    The term "equals" rather than the term "overlaps" for example is merely an unfortunate consequence of LINQ being designed to use keywords that are familiar to SQL developers. I've already made the case to add LINQ keywords in C# vNext, and this is another perfect example of where additional keywords would be quite useful for Rx devs.

    Take the following query for example:

    left.Join(right, leftDuration, rightDuration, (l, r) => new { l, r })

    And in query comprehension syntax:

    from l in left
    join r in right
    on leftDuration equals rightDuration
    select new { l, r }

    Let's use the hypothetical overlaps keyword instead.

    from l in left
    join r in right
    on leftDuration overlaps rightDuration
    select new { l, r }

    It's using 4 point events (i.e., notifications without any time semantics) to form two windows (i.e., durations), and projects pairs of notifications that occur while their durations are both available (i.e., they overlap). Note that we're not expecting the point events themselves to happen simultaneously (that would be absurd), so we're creating durations and comparing them instead. All notifications within the right duration that occur while a given left duration is still available are projected into pairs. Furthermore, this happens for every overlapping left duration, thus each right notification may be projected into multiple left windows.  Wes Dyer once described this operator as something like "creating windows on the left side and pushing values on the right side into those windows, when the values on the right occur while a window on the left is still open". The right side has a duration too though, meaning that you could look at it both ways; i.e., left values are projected into windows created on the right. It goes both ways. That's "join" semantics.

    left = point event; opens a window
    right = point event; opens a window
    leftDuration = point event; closes left window; forms the duration [left...leftDuration]
    rightDuration = point event; closes right window; forms the duration [right...rightDuration]

    Therefore, leftDuration equals rightDuration actually means leftDuration overlaps rightDuration, as in:

    "Give me all left and right values that occur at the same time, whereby 'time' is not their point events but the durations we've assigned."

    Does that help at all?

    - Dave


    http://davesexton.com/blog

    • Edited by Dave Sexton Monday, September 29, 2014 9:55 AM
    Monday, September 29, 2014 9:53 AM
  • Thanks. Yes, 'LINQ equals' == 'RX overlaps' makes it clear.

    Thanks again. You have been very helpful.

    Tuesday, September 30, 2014 2:41 AM