Strange crash in OnObserve

Question

Getting an exception I'm having trouble debugging. It seems to happen when an exception is thrown via Observable.Start on a TaskPool thread, then the IObservable is observed on the Dispatcher scheduler. Eventually this brings down the app (The exception thrown in the Start is *not* this one) - I've also got an OnError handler on the Subscribe:

0:000> !pe
Exception object: 046ce908
Exception type:   System.NullReferenceException
Message:          Object reference not set to an instance of an object.
InnerException:   <none>
StackTrace (generated):
    SP       IP       Function
    0014ECB4 086298E4 System_Reactive_6880000!System.Reactive.Stubs.<.cctor>b__1(System.Exception)+0x4
    0014ECB8 080577DB System_Reactive_6880000!System.Reactive.AnonymousObserver`1[[System.__Canon, mscorlib]].Error(System.Exception)+0xb
    0014ECBC 080576AC System_Reactive_6880000!System.Reactive.AbstractObserver`1[[System.__Canon, mscorlib]].OnError(System.Exception)+0x1c
    0014ECC8 080577AF System_Reactive_6880000!System.Reactive.AutoDetachObserver`1[[System.__Canon, mscorlib]].Error(System.Exception)+0x2f
    0014ECDC 080576AC System_Reactive_6880000!System.Reactive.AbstractObserver`1[[System.__Canon, mscorlib]].OnError(System.Exception)+0x1c
    0014ECE8 081B920A System_Reactive_6880000!System.Reactive.ScheduledObserver`1+<>c__DisplayClassf[[System.__Canon, mscorlib]].<error>b__d()+0x32
    0014ECFC 081B9005 System_Reactive_6880000!System.Reactive.ScheduledObserver`1[[System.__Canon, mscorlib]].<ensureactive>b__5(System.Action)+0xa5</ensureactive></error></none>

OS Thread Id: 0x660 (0)Child SP       IP Call Site
0014ebf8 760fb727 [HelperMethodFrame: 0014ebf8] 
0014ecb4 086298e5 System.Reactive.Stubs.<.cctor>b__1(System.Exception)
0014ecb8 080577db System.Reactive.AnonymousObserver`1[[System.__Canon, mscorlib]].Error(System.Exception)
0014ecbc 080576ac System.Reactive.AbstractObserver`1[[System.__Canon, mscorlib]].OnError(System.Exception)
0014ecc8 080577af System.Reactive.AutoDetachObserver`1[[System.__Canon, mscorlib]].Error(System.Exception)
0014ecdc 080576ac System.Reactive.AbstractObserver`1[[System.__Canon, mscorlib]].OnError(System.Exception)
0014ece8 081b920a System.Reactive.ScheduledObserver`1+<>c__DisplayClassf[[System.__Canon, mscorlib]].<error>b__d()
0014ecfc 081b9005 System.Reactive.ScheduledObserver`1[[System.__Canon, mscorlib]].<ensureactive>b__5(System.Action)
0014ed40 00fa67bf System.Reactive.Concurrency.Scheduler.<schedule>b__0(System.Action`1<system.action>, System.Action`1<system.action`1<system.action>>)
0014ed54 00fa675b System.Reactive.Concurrency.Scheduler+<>c__DisplayClassb`1[[System.__Canon, mscorlib]].<invokerec1>b__8(System.__Canon)
0014ed68 00fa6685 System.Reactive.Concurrency.Scheduler.InvokeRec1[[System.__Canon, mscorlib]](System.Reactive.Concurrency.IScheduler, Pair`2<system.__canon,system.action`2<system.__canon,system.action`1<system.__canon>>>)
0014ed88 08628eb6 System.Reactive.Concurrency.DispatcherScheduler+<>c__DisplayClass1`1[[System.Reactive.Concurrency.Scheduler+Pair`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]], System.Reactive]].<schedule>b__0()
0014ed98 6f86e33d System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
0014edb8 6f86e20a MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
0014edfc 6f86e085 System.Windows.Threading.DispatcherOperation.InvokeImpl()
0014ee34 6f86dfc8 System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(System.Object)
0014ee3c 721be1b7 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0014eea8 721be0f6 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0014eebc 721be0b1 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
0014eed4 6f86de9b System.Windows.Threading.DispatcherOperation.Invoke()
0014ef0c 6f86ca6b System.Windows.Threading.Dispatcher.ProcessQueue()
0014ef50 6f86cc1a System.Windows.Threading.Dispatcher.WndProcHook(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
0014ef9c 6f86e50b MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
0014efd8 6f86e45b MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
0014efe8 6f86e2d6 System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
0014f008 6f86e20a MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
0014f04c 6f86e085 System.Windows.Threading.DispatcherOperation.InvokeImpl()
0014f084 6f86dfc8 System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(System.Object)
0014f08c 721be1b7 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0014f0f8 721be0f6 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0014f10c 721be0b1 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
0014f124 6f86de9b System.Windows.Threading.DispatcherOperation.Invoke()
0014f15c 6f86c4ba System.Windows.Threading.Dispatcher.InvokeImpl(System.Windows.Threading.DispatcherOperation, System.Threading.CancellationToken, System.TimeSpan)
0014f1ac 6f86c44c System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
0014f1e4 6f86d931 MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)
0014f368 0022a0b9 [InlinedCallFrame: 0014f368] 
0014f364 6f886ac8 DomainBoundILStubClass.IL_STUB_PInvoke(System.Windows.Interop.MSG ByRef)
0014f368 6f86bdcf [InlinedCallFrame: 0014f368] MS.Win32.UnsafeNativeMethods.DispatchMessage(System.Windows.Interop.MSG ByRef)
0014f39c 6f86bdcf System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
0014f3e8 6f86bae1 System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame)
0014f3f4 6f85a17b System.Windows.Threading.Dispatcher.Run()</schedule></system.__canon,system.action`2<system.__canon,system.action`1<system.__canon></invokerec1></system.action`1<system.action></system.action></schedule></ensureactive></error>

Answer 1

Hi Paul,

Have you tried breaking on first-chance CLR exceptions and disabling Just My Code?  What is the stack trace at that point?

- Dave

---

http://davesexton.com/blog

Answer 2

So, this exception is definitely *originating* from my application - it's an exception thrown inside an Observable.Start which marshals it into the OnError for the IObservable. This happens correctly. However, the next exception I get is this one, whereas I would expect my OnError to be called in the WPF Dispatcher context.

Answer 3

Hi Paul,

To be clear, what you meant by "(The exception thrown in the Start is *not* this one)" and "However, the next exception I get is this one" is that the NullReferenceException is being thrown by the framework's code, not by your code?

The stack trace that you posted doesn't seem to be for a first-chance exception.  Can you get Visual Studio to break immediately when NullReferenceException is thrown to see where it's originating from?

- Dave

---

http://davesexton.com/blog

Answer 4

So, here's the simplest version of what *appears* to be happening:

var foo = Observable.Start(() => {
    // This 1st chance exception happens correctly, and is caught 
    // by Observable.Start. All is well!
    throw new FooBarException("Aieeeeee");
}, Scheduler.TaskPoolScheduler);

// ...then it gets here, and we suddenly get NullReferenceException
foo
    .ObserveOn(DispatcherScheduler.Current)
    .Subscribe(_ => {}, ex => Console.WriteLine(ex));

This appears to be the next time an Exception throws *after* the FooBarException. Unfortunately in this case, it's difficult for me to repro because in our app it only happens on clean machines (i.e. not my dev machine)

Answer 5

Hi Paul,

It looks like your code is throwing an exception, then sometime before it's observed a bug in Rx is throwing a NullReferenceException, catching the buggy exception and then re-throwing it on the Dispatcher, without passing it to your registered OnError handler.  And your concerend with why the error isn't reaching your registered handler and is instead crashing your application.  Is that correct?

Perhaps it makes sense that the NullReferenceException isn't being handled because it represents a bug in Rx that should be considered fatal.  But that wouldn't be my primary concern anyway.  I'd like to know where the NullReferenceException is originating from so that perhaps you can workaround it.

If you can't repro locally and therefore can't break on a first-chance NullReferenceException, then try using adplus.vbs or another tool to attach a JIT debugger that takes a mini crash dump when NullReferenceException is thrown on a clean machine.

http://support.microsoft.com/kb/286350

Note the section on First chance exceptions.

http://support.microsoft.com/kb/105675

- Dave

---

http://davesexton.com/blog

Answer 6

Alright, I'm hitting a similar version of this bug. This time, the original exception is rethrown on the UI thread instead of OnError'ing. So, in summary:

var foo = Observable.Start(() => {
    // This 1st chance exception happens correctly, and is caught 
    // by Observable.Start. All is well!
    throw new FooBarException("Aieeeeee");
}, Scheduler.TaskPoolScheduler);

// ...then it gets here, but instead of Console.WriteLine being called,
// FooBarException gets rethrown on the UI thread.
foo
    .ObserveOn(DispatcherScheduler.Current)
    .Subscribe(_ => {}, ex => Console.WriteLine(ex));

Answer 7

Hi Paul,

Where is DispatcherScheduler.Current defined?  I only see DispatcherScheduler.Instance.  Are you using the latest version of Rx?

- Dave

---

http://davesexton.com/blog

Answer 8

Hi Paul,

Also, are you sure that DispatcherScheduler.Current isn't returning null?

- Dave

---

http://davesexton.com/blog

Answer 9

And that foo is not null too.