How to cancel socket operations

Question

I have an application that acts as an IPC mechanism between two systems running at different sites.  There are multiple sockets being used with two threads per socket to handle sending and receiving.  When a glitch occurs, the socket in error is closed and a new connection is opened.

Now, I have a problem in that the next socket opened may get the same handle as the previous.  That creates all sorts of confusion between the old threads, which should be exiting, and the new threads that think they have exclusive use of the socket.  To get around this issue, I'm now performing a shutdown on the old socket and placing it on a stack for later clean-up.  That ensures that the new socket(s) get unique handles.  One issue though, is that the old threads don't seem to know that the socket is being shutdown.  The closesocket call would cause them to be notified immediately, but the shutdown call doesn't do this.  If there is a recv or send call in progress, the old thread can sit there for over a minute before erroring out.  I've tried setting SO_RCVTIMEO, SO_SNDTIMEO, and FIONBIO, but they don't seem to have any affect on operations already issued.

Is there some mechanism whereby I can cancel operations going on in these other threads?  Alternatively, is there some mechanism whereby I can keep the system from reusing socket handles right away?

Randy

Answer 1

OK - I've figured this out for myself.  I've converted all send() and recv() calls to WSASend() and WSARecv() and provided WSAOVERLAPPED structures.  If the WSASend() or WSARecv() call returns zero, the operation completed immediately the same as the old send() and recv() logic.  If it returns non-zero, I check for WSA_IO_PENDING and then call WSAGetOverlappedResult() with a wait.  If I need to shutdown the a socket, I can simply signal the event specified in the WSAOVERLAPPED structure.  That effectively cancels any operation in progress.

Answer 2

Well, I guess this will teach me not to believe everything I read.  It appears that the WSARecv() operation isn't canceled, nor does it get canceled when the thread exits.  The operation seems to hang around for a bit, and then gets an access violation in the TCP/IP thread.  At least that's what I think is going on.

When I perform the WSAStartup(), a thread is created for TCP/IP to fool around in.  It appears to be this thread that is crashing sometime after my threads exit.  I presume that it's the Overlapped I/O causing the problem, as my only problem before was the delay in waiting for my threads to wakeup and exit.  A crash is a much bigger problem, so I'm going to have to back out these changes.  It looks like I'm back at square one.  Gack!

 

Answer 3

Close the socket to cancel a pending operation.  Since you cannot be sure how much data has been sent / received or not, this approach should be used.

Answer 4

Hello,
I have experienced the same king of problem. The sequence, on the client side (Socket; ..., ShutDown, CloseSocket, Socket) gives the same socket number at the second socket instruction. In my case, I use overlapped IO with completion routine and the second (new with the old number) receives a call to the completion routine before any ReadFileEx or WriteFileEx on whatever side. The error code is 0, the lenght is small but not zero and of course there is no overlapped data structure attached to it so it goes in acces_violation.

Thank to your post, I inserted a timer call (1/100) between the CloseSocket and Socket instruction and everything went fine. The nex socket has a different number. As I reuse connections I do not care of the small delay on the client side.

This test was done on XP-HE, SP1, with loopback.

Answer 5

Are you calling PostQueuedCompletionStatus to post your own messages to the completion port? Once you've closed the first socket handle it is disassociated from the completion port such that the next socket (with the same handle value) will not be associated (i.e. overlapped operations on the second handle will not use the completion port). Also, Winsock would not complete an overlapped operation without giving the pointer to the WSAOVERLAPPED structure used to initiate the operation.

Of course, if you aren't calling GetQueuedCompletionStatus in a timely fashion you may end up retrieving an completion packet from the completion port after you've already created the second socket handle.

Answer 6

The manual says to either just close the socket or call CancelIo() and then close the socket. In either case, the completion routine will be called once more with the error code WSA_OPERATION_ABORTED, and only after that you can free the buffers.

If you want to make sure that it gets called right now, you can do the following:

CancelIo((HANDLE)socket);

SleepEx(0, TRUE); // the completion will be called here

closesocket(socket);