locked
Async Classes Implementing IDisposable

    Întrebare

  • What's the current thought process on implementing async classes that need something like IDisposable? There are plenty of examples from the framework where the async methods could render IDisposable obsolete / problematic.

    • System.Stream
    • System.Net.TcpClient
    • System.Net.HttpClient
    • System.Net.Mail.SmtpClient

    The last of these is a particularly effective illustration because a well-behaved SMTP client needs to send a QUIT message before closing the connection. It has to send a QUIT message, receive the server confirmation, and then shutdown the network stream. The server's end of the network stream needs to be disposed like a local resource.

    So what is the right way to implement a class that is both Disposable and supposed to be providing an asynchronous behavior?

    1. Dispose could just block the calling thread until all the async cleanup operations complete and it can properly dispose of it's resources.
    2. Some sort of reference counted like behavior where dispose won't actually do anything until all async operations have released their resources? The async calls would need to check for and dispose themselves if flagged like this. Maybe this could work with the existing apis but is also going to be messy and error-prone.
    3. Behave badly and just kill the resources we can kill without blocking. (This could be likened to not disposing some resources, for example the network stream on the SMTP server)
    4. We could just tell people using the library to not call dispose on anything anymore and leave all the resources unclaimed until GC or use a custom 'disposal' scheme invented by each library developer
    5. Maybe there is some new support built into the language / framework for asynchronous disposal that i just haven't heard about?

    For me the primary concern is how an async client is supposed to cleanup resources that would best be disposed of asynchronously. I have seen elsewhere answers like don't do that, but like in the SMTP Client example, there is some overhead involved in required cleanup that does warrant such a thing.





    • Editat de hannasm 19 februarie 2012 01:38
    19 februarie 2012 01:05

Răspunsuri

  • This has always been a problem with asynchronous programming. Ideally, asynchronous programming would be purely functional, but that's seldom the case in the real world. IDisposable is just one example of (functional) async having a minor conflict with (stateful) OOP.

    I think you're actually addressing a couple of issues here:

    1. If a resource is disposed with outstanding asynchronous operations, what happens to the operations?
    2. How should resource disposal be handled if it requires long-running logic (which should really be asynchronous)?

    The historical approach for issue (1) has been to to end the asynchronous operation with an error. File handles are an example of this. If you have an unmanaged file handle open with an overlapped I/O operation going, and you close the file handle, then the overlapped I/O operation completes with an error. I would expect the async Stream read/write operations to behave the same way.

    The historical approach for issue (2) is to change the meaning of Dispose/Close. WinSock socket handles are an example of this. TCP/IP socket closure is quite complex, consisting of a 4-way handshake for a graceful disconnect (A sends FIN; B sends ACK; B sends FIN; A sends ACK). When an application closes a socket handle, what actually happens is the WinSock DLL takes responsibility for the socket, keeping it alive until the (asynchronous) shutdown sequence completes. So the application is not even aware the socket connection is still technically open. (This is what happens by default; the application can set socket options if it wants to control the shutdown behavior, e.g., have the close actually close the socket handle immediately).

    In my experience, the IDisposable issue in particular doesn't come up much during async development. Much of the time, async code will use an IDisposable resource, but not be a part of an IDisposable class.

    (re 1) If you did need an async method on an IDisposable class, then you could follow the common practice: have all outstanding asynchronous operations complete with error. (Of course, there's a race condition between success and error, which can be ignored). Or you could just ignore the problem, and call it "undefined behavior" (this is the option I have taken in the past). Either way, the recommendation for client code would be: cancel or await all asynchronous operations for a given object before disposing that object. I think that developers would expect that kind of restriction.

    (re 2) For long-running Dispose methods, I would just have the Dispose method start the asynchronous shutdown logic. Eventually, the shutdown logic will complete, and the actual underlying resource should be disposed at that point. Depending on the resource, you may want to provide shutdown options or an async Shutdown method. (e.g., an extra socket handle hanging around for a few seconds doesn't impact most apps, but an extra file handle hanging around can cause problems - client code needs to know when an actual file handle is closed).

    With these approaches, you end up with consistency for client code:

    • IDisposables should always be disposed.
    • Asynchronous operations should be completed before Dispose is called.
    • Dispose won't block for a long time. If necessary, shutdown logic will happen in the background and will be invisible to client code.
    • If client code does need to know when shutdown completes, then shutdown options or an async Shutdown method is provided.

    IMO. :)

          -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible


    24 februarie 2012 12:19

Toate mesajele

  • This has always been a problem with asynchronous programming. Ideally, asynchronous programming would be purely functional, but that's seldom the case in the real world. IDisposable is just one example of (functional) async having a minor conflict with (stateful) OOP.

    I think you're actually addressing a couple of issues here:

    1. If a resource is disposed with outstanding asynchronous operations, what happens to the operations?
    2. How should resource disposal be handled if it requires long-running logic (which should really be asynchronous)?

    The historical approach for issue (1) has been to to end the asynchronous operation with an error. File handles are an example of this. If you have an unmanaged file handle open with an overlapped I/O operation going, and you close the file handle, then the overlapped I/O operation completes with an error. I would expect the async Stream read/write operations to behave the same way.

    The historical approach for issue (2) is to change the meaning of Dispose/Close. WinSock socket handles are an example of this. TCP/IP socket closure is quite complex, consisting of a 4-way handshake for a graceful disconnect (A sends FIN; B sends ACK; B sends FIN; A sends ACK). When an application closes a socket handle, what actually happens is the WinSock DLL takes responsibility for the socket, keeping it alive until the (asynchronous) shutdown sequence completes. So the application is not even aware the socket connection is still technically open. (This is what happens by default; the application can set socket options if it wants to control the shutdown behavior, e.g., have the close actually close the socket handle immediately).

    In my experience, the IDisposable issue in particular doesn't come up much during async development. Much of the time, async code will use an IDisposable resource, but not be a part of an IDisposable class.

    (re 1) If you did need an async method on an IDisposable class, then you could follow the common practice: have all outstanding asynchronous operations complete with error. (Of course, there's a race condition between success and error, which can be ignored). Or you could just ignore the problem, and call it "undefined behavior" (this is the option I have taken in the past). Either way, the recommendation for client code would be: cancel or await all asynchronous operations for a given object before disposing that object. I think that developers would expect that kind of restriction.

    (re 2) For long-running Dispose methods, I would just have the Dispose method start the asynchronous shutdown logic. Eventually, the shutdown logic will complete, and the actual underlying resource should be disposed at that point. Depending on the resource, you may want to provide shutdown options or an async Shutdown method. (e.g., an extra socket handle hanging around for a few seconds doesn't impact most apps, but an extra file handle hanging around can cause problems - client code needs to know when an actual file handle is closed).

    With these approaches, you end up with consistency for client code:

    • IDisposables should always be disposed.
    • Asynchronous operations should be completed before Dispose is called.
    • Dispose won't block for a long time. If necessary, shutdown logic will happen in the background and will be invisible to client code.
    • If client code does need to know when shutdown completes, then shutdown options or an async Shutdown method is provided.

    IMO. :)

          -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible


    24 februarie 2012 12:19
  • Thanks for the feedback.

    The two cases are definitley related, but from my perspective deciding when dispose should be called is something that is basically left as a challenge to the client wheras case 2 has to be addressed by the library, even though it could (in some perfect world) also be reasonably left as a question for the client to decide.

    For case 1 asynchronous operations will end-up giving errors or undefined behavior when the underlying resources are not valid. This is a challenge that really hasn't become any more complicated because of asynchronous programming. Remote/Unmanaged resources becoming unavailable at seemingly random times is perhaps less likely than a premature dispose but generally has the same affect. I think there is possibly an opportunity to handle the disposing case more cleanly (maybe through cancellation tokens) but aside from providing a more friendly API is there really much difference between a resource closing remotely and a resource that is closed by another thread locally?

    On the other hand, case 2 does present challenges to a library designer because there are scenarios where dispose needs to wait longs periods before completing. The disconnect behaviors such as in SmtpClient or a tcp socket is one scenario, and the asynchronous operations in the case 1 above is a second.

    Without compiler support for an asynchronous dispose, spawning a new thread to complete long running resource cleanup does strike me as a good approach. If an 'async using()' syntax is ever added to the language, you would probably have a simple option of returning the background dispose task instead of scheduling it yourself. I still have some questions running through my mind about doing this though:

    • Do you run this background thread with TaskFactory.Start or do you need to take a scheduler parameter, it seems like somebody will want to choose the scheduler for their dispose tasks eventually?
    • Is it safe for the object to be disposed and still access it in your background thread? Sure you won't kill the resources in the dispose call, but will there be any side-effects from accessing a disposed object, that are outside library developer control, and will invalidate the object before the background thread gets to complete?
    • How is this going to affect a program shutdown? If you dispose your async class, and then exit from Main() will the background thread still be given the time it needs to finish? If not what extra steps can be taken to deal with that?

    With an 'async using()' i don't think you have to answer these questions ahead of time, and a client could more cleanly deal with dispose, whichever way they want to.

    25 februarie 2012 00:15