Using Reactive Extension Buffer RRS feed

  • Question

  • I am trying to use Rx and its Buffer to search for files but UI gets frozen. It does not seem to do as described.

    ...each element of an observable sequence into a buffer that’s sent out when either it’s full or a given amount of time has elapsed.

    So I expected on local drive listbox will get 100 items at a time as it is faster than .2s, while on slow network it will output whatever buffer collects within .2 seconds (as long as it is less than 100 items). I tried enumerating on other thread while observing must obviously be on dispatcher.

    Is there something I am doing wrong here?

    IDisposable observer;
    IObservable<IList<FileInfo>> bufferedFiles;
    ObservableCollection<object> _fileCollection = new ObservableCollection<object>();
    public void EnumerateFiles(string myfolder, string filter)
        var syncContext = SynchronizationContext.Current;
        DirectoryInfo dir = new DirectoryInfo(myfolder);
        this.bufferedFiles =
         Observable.Buffer(dir.EnumerateFiles(filter, System.IO.SearchOption.AllDirectories)
        .ToObservable(NewThreadScheduler.Default), TimeSpan.FromSeconds(.2), 100, NewThreadScheduler.Default)
    = this.bufferedFiles.Subscribe(outputFiles);
    private void outputFiles(IEnumerable<FileInfo> FI)
        foreach (var file in FI)

    • Moved by Kristin Xie Monday, October 6, 2014 1:33 AM rx related
    Friday, October 3, 2014 2:48 PM

All replies

  • Hi friend,

    Like your title mentioned, this is a Reactive Extension case. There is a special forum for Rx. So i moved your thread to

    Data Platform Development >Reactive Extensions (Rx)  forum for better support.

    Have a nice day!


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, October 6, 2014 1:33 AM
  • My guess is that your SynchronizationContext is not set properly. If you pass Scheduler.Default to ObserveOn instead of your syncContext you get the proper output.


    A good series of articles on SynchronizationContexts is here:

    Friday, October 10, 2014 5:47 PM
  • If I do that I get

    This type of CollectionView does not support changes to its SourceCollection from a thread different from the Dispatcher thread.

    Friday, October 10, 2014 5:53 PM
  • Ah ok so when your call to OnNext modifies the _fileCollection collection then it throws that exception

    In that case you can use ObserveOnDispatcher() instead of the standard ObserveOn() method using the System.Reactive.Linq namespace when referencing the System.Reactive.Windows.Threading assembly.

    If you need to use the syncContext you can only use SynchronizationContext.Current if you are calling EnumerateFiles from the Dispatcher thread, as a result of an event callback from the UI, like a button's Click event or the forms Loaded event.  Then syncContext should be populated correctly.  If you're trying to call EnumerateFiles in the constructor for the form you will have to do it after form initialization or the syncContext will not have been created yet.
    Friday, October 17, 2014 10:47 PM
  • ObserveOnDispatcher() eliminates the error but UI is still completely unresponsive. Buffer does not seem to be fit for this kind of work.
    Wednesday, October 22, 2014 3:51 PM
  • If you wrap the code you provided in a console project and call EnumerateFiles from Main, supplying Scheduler.Default to the ObserveOn method, you'll see that the code does function and that buffer is doing its work properly.  If you supply a fabricated SynchronizationContext then the output will hang like you're describing.  This leads me to believe that it's an issue with the threads and not with buffer construct itself, possibly a locking issue.

    I don't quite know yet where the issue is coming from, but that's where I'd focus my time.

    Wednesday, October 22, 2014 5:39 PM
  • Yes, it seems it is working properly, but unfortunately it does not seem like it is appropriate for populating WPF ListBox with enumerated files.
    Wednesday, October 22, 2014 5:52 PM
  • ok so I went over your code again with a UI to provide a good sync context and it looks like your code works for updating the collection, but the listbox may not be listening to the collection changed event on _fileCollection.  If you change it to update the listbox.items itself then you should notice it works with the sync context.
    Wednesday, October 22, 2014 6:45 PM
  • This way I would loose all benefits of ObservableCollections. The problem I see is that for each buffer output so many INotifyChanged events are called that ListBox stops responding. I could implement custom AddRange with reset event, but rebinding will introduce more problems, like loosing selection, and it still there are no guarantees that it would be responsive.
    Thursday, October 23, 2014 7:15 PM
  • The only way I can see to test that hypothesis is to programmatically add a single item to the collection instead of subscribing to the buffer, or possibly before subscribing to the buffer.  That way you are sure that at least one item is being added to the collection, and by extension the ListBox, synchronously.  If this object (a simple test string should work) doesn't make it to the list then you have a more fundamental problem than introducing asynchrony.

    Edit: grammatical correction.

    Monday, October 27, 2014 7:39 PM
  • Even with buffer it works if there is not a lot of files... for example if I use


    enumeration is quick and I get results in ListBox right away.

    But as soon I set it to search whole drive WPF ListBox can not handle the incoming number of files. For example, in the first second it gets 20K files and ListBox hangs. Files are keep coming and until all files are enumerated whole program is frozen.

    I would rather it shows 30 files, refresh, show another 30... and so on while letting user browse files the whole time with responsive UI. Number of files should be system dependent, so $200 tablet should show less files than 5K workstation.

    Monday, October 27, 2014 7:49 PM
  • I see, so you're still able to see updated collection counts in the debug output, but you're unable to navigate the ListBox while it's still updating.  This I believe may be due to the way ListBox implements the event handler responsible for responding to CollectionChangedEvents.  If the box redraws every item in the control on every change to the collection, and there are several changes made to the collection every second, then it could be causing your box to appear frozen when in reality it's actually refreshing with each change.  Either way you will have to figure out a way around making so many changes at once or at least to restore responsiveness to the box without waiting for the entire dataset to come back.

    This URL has an interesting take on the subject, though might not be what you're looking for.  I'll see what else I can find. (sorry I still can't post links yet, for whatever reason)

    There is also an Attached Property that can be used with ListBox to make it only draw the items that will be displayed on the screen called VirtualizingStackPanel.IsVirtualizing that might make each refresh a little easier on the UI thread.

    Monday, October 27, 2014 8:09 PM