none
system.io.file.delete (and some other) functions not completing execution before next command RRS feed

  • Question

  • I have a problem on VB.NET DevStudio 2010 (latest Framework ver 4), whereas the SYSTEM.IO.FILE.DELETE is not reporting any errors when executing (i.e. yes, file should be assumed to be deleted), however, upon checking with SYSTEM.IO.FILE.EXISTS, it's still there for a couple of seconds before it's gets removed from the filesystem, which is a big problem since the DELETE should really ensure that the file is getting deleted before returning the control to my code to give the feeling that it's been deleted.

    The code goes like this:

     

    Function CheckAndCorrectFile(FileName as string)
     If system.io.file.exists (FileName) then
      Try
       Dim FS as System.IO.FileStream   = New System.IO.FileStream(FileName, IO.FileMode.OpenOrCreate, IO.FileAccess.Write, IO.FileShare.None)
       Dim BR as System.IO.BinaryReader = New System.IO.BinaryReader(FS)
       Dim Buffer(&H10) as byte

       Buffer = BR.ReadBytes(&H10)

       BR.Close
       FS.Close

       BR = Nothing
       FS = Nothing
      Catch
       Msgbox "Error...."
      End Try

     Else
      Try
       Dim FS as System.IO.FileStream   = New System.IO.FileStream(FileName, IO.FileMode.OpenOrCreate, IO.FileAccess.Write, IO.FileShare.None)

       Dim BW as System.IO.BinaryWriter = New System.IO.BinaryWriter(FS)
       Dim Buffer() as byte = {&H48,&H45,&H49,&H44,&H45,&H52,&H20,&H53,&H41,&H54,&H49}

       BW.Write(Buffer,0,Buffer.Length)

       BW.Close
       FS.Close

       BW = Nothing
       FS = Nothing
      Catch
       Msgbox "Error...."
      End Try
     End if

    End Function

     

    Sub Main
     CreateAFile(FileName)

     Dim FileName as string = "D:\MyDir\1234\Heider.TXT"

     If system.io.file.exists (FileName) then
      Try
       System.Io.File.Delete (FileName)
       CheckAndCorrectFile(FileName)
      Catch
       Msgbox "Error...."
      End Try
     End if
     
    End Sub

    ...

     

    Now, If I debug the code line-by-line, then it works fine, Nothing wrong, the file gets created, removed, and then created again.

     

    However, if I run this in fast-motion (i.e. RUN not DEBUG), then I get an error (sometimes) on the following line:

     

     Buffer = BR.ReadBytes(&H10)

     

    When looking back, it seems the code has gone through the logic, deleted the FileName, but when gone into the function, the file was still not removed from the FileSystem, which made the function of EXISTS successful, finally upon opening the file, and then tried to READ, it then failed as the filesystem "then" deleted the file in somehow like a background thread or something and not ensuring the file was deleted before the function executed.

     

    To fix the problem, I put the following loop, the fix isn't great, which is why I am asking, "has anyone seen this problem before", and also "is there anyway to ensure that the filesystem (complete's) what it's been told to do, or FLUSH in other words before it hand over the control back to my next line of execution"?

     

    The code to fix is this:

    After the following line:

       System.Io.File.Delete (FileName)

     

    You have to put the following loop

    Do While System.IO.File.Exists(FileName)
     console.writeline("SaveArrayCache-Waiting for FileDeletion")
     Threading.Thread.Sleep(1)
    Loop

     

    This fix works, and you can see in the CONSOLE that few times the message of "Waiting for ..." and then it continues without errors.

     

    I am posting this because I am seeing more of the same problem with different SYSTEM.IO.FILE functions, as if these functions are executing in a "background thread" instead of ensuring they complete before returning the control back.

     

    It would be nice to know what you think?

     

    Regards

    Heider Sati

     

     
    Heider Sati
    Friday, August 19, 2011 6:21 PM

Answers

  • Hi Jader3rd

    Yes, the .Write is a type on this forum, the code actually does have .READ, the reason is because I cannot cut-and-paste from my development into here as it's a lot more than just these commands.

    Now, regarding the code, it seems that when the code is running in foreground mode, this issue does not occur, therefore, the fix to this problem was to change from "MyThread.IsBackground" from "True" into "False", all worked fine, and I ran about 100 times with no issues which is great.

    I also considered using the Windows SDK (CreateFile), this works flawlessly with or without a background thread.

    Rudedog2: I am aware of the I/O taking a long time, but comparing SYSTEM.IO.DELETE with the Windows SDK's DELETE, it's a lot different, I think you are slightly missing the point, regardless of the I/O taking time, when calling the SYSTEM.IO.DELETE to use this I/O, it should wait for that I/O operation to complete, which can take seconds, or days, I don't expect (as a developer) that doing a DELETE that takes (say; an hour) to finish, for the command to execute in half-a-second telling me (ok, deleted) when the file actually has not been removed from the FS yet. and I think this is the problem.

    Anyhow, I fixed this in 3 ways:

    1) Either using the ugly Sleep/Loop above. Or;

    2) Replacing the SYSTEM.IO.FILE with the internal (non-wrapper) Windows File I/O SDK. Or:

    3) Best so far: Changing the thread from executing in the Background to a Foreground mode. This so far, does the trick, the price to pay here is CPU has to pay more attention, which I think is ok for today's systems anyway.

     

    I hope you (or all) found this useful, I reported this to Microsoft already, it's being investigated, and hopefully, a fix might be out at some point, who knows!

     

    Kind Regards

    Heider

     

     


    Heider Sati
    • Marked as answer by Heider Saturday, August 20, 2011 10:21 PM
    Saturday, August 20, 2011 10:21 PM

All replies

  • I wouldn't like that loop in my code, either.  Any form of I/O can be finicky at times.  You have to allow for the unexpected.  You might try File.Move to another folder first, and then delete it from there.  File moves are quick.  I believe that some File operations occur on separate threads.

    Rudy   =8^D


    Mark the best replies as answers. "Fooling computers since 1971."

    http://thesharpercoder.blogspot.com/

    Friday, August 19, 2011 8:06 PM
  • Delete is going to be as fast, if not faster than move, because all delete does is remove the entry of the file from the directory system. A move (if the move is to the same hard drive) changes the directory system, but the bytes stay in the same place on the disk.

    As for why you might be getting errors in a tight loop like that? My first suspect would be an anti virus and my second would be a file system indexer.

    Friday, August 19, 2011 8:18 PM
  • Not knowing how antivirus or the indexer work, I would have suspected about file caching operations, thinking that AV and indexer intrusions were serialized by a driver.  But then again, I stress the fact that I don't know how these really work.
    MCP
    Friday, August 19, 2011 9:01 PM
  • Delete is going to be as fast, if not faster than move, because all delete does is remove the entry of the file from the directory system. A move (if the move is to the same hard drive) changes the directory system, but the bytes stay in the same place on the disk.

    As for why you might be getting errors in a tight loop like that? My first suspect would be an anti virus and my second would be a file system indexer.


    I was not implying that Move was faster than Delete.  I was pointing out that by the time the FileCheck occurs, the file will be gone.

    Rudy   =8^D


    Mark the best replies as answers. "Fooling computers since 1971."

    http://thesharpercoder.blogspot.com/

    Friday, August 19, 2011 9:24 PM
  • Hi Guys,

    thanks for replying,

    I know about MOVE and DELETE, the point that I am trying to make isn't about which one is faster, it's about why when I say "DELETE", it comes back saying "OK, continue your code next line.." but I still didn't finish my DELETE command, When my next line is processing things assuming that DELETE did it's jub, only to be shocked that it hasn't.

    By the way, the problem is seen more when the code I posted above, runs from a Background Thread.

    I've been developing for at least 15 years now, therefore, No indexing is turned on, and Anti-Virus, isn't coming near that folder. The problem is to do with the filesystem being used to buffer socket data which are in and out very fast.

    I just want to say, I've never seen this problem before, can't find any property under the system.io.file to say something like "ensure the command that I told you to execute is COMPLETED!"... i.e. something like "FS.FLUSH" of "BW.FLUSH", but nothing like it.

    Very annoying having to workaround the problem using a SLEEP loop!.

    Kind Regards

    Heider

     


    Heider Sati
    Saturday, August 20, 2011 4:24 AM
  • Like I said, I wouldn't like using a hack like SLEEP, either.  I suggested a different hack.  If you are using threads, then you should be aware that you need to signal other threads when the thread's task is completed. 

    You are dealing with I/O, which means that tasks can take random lengths of time to complete.  That is why you see mechanisms built into I/O operations like timeout values.

    Rudy   =8^D


    Mark the best replies as answers. "Fooling computers since 1971."

    http://thesharpercoder.blogspot.com/

    Saturday, August 20, 2011 1:04 PM
  • You said this happens sometimes, about how often does it happen?

    What type of disk storage system is this running on (or does that not matter)?

    Have you tried running Procmon to see all of the accesses to the file in question?

    Is it possible you could have two background threads working on the same file?

     

    The reason why I ask about the disk storage system is that big enterprise systems will have quite a complex layer of caches inbetween the OS and the physical disk. I've run into issues their drivers having bugs in more than one occasion.

    Saturday, August 20, 2011 5:13 PM
  • In the example code you're opening the file with  IO.FileAccess.Write, but then trying to read. You're not opening the file with read permissions. Besides, what's the exception you're getting?
    Saturday, August 20, 2011 5:26 PM
  • I just ran this code 10 times and never hit an issue. The code never went down the read path.
    Saturday, August 20, 2011 5:33 PM
  • Hi Jader3rd

    Yes, the .Write is a type on this forum, the code actually does have .READ, the reason is because I cannot cut-and-paste from my development into here as it's a lot more than just these commands.

    Now, regarding the code, it seems that when the code is running in foreground mode, this issue does not occur, therefore, the fix to this problem was to change from "MyThread.IsBackground" from "True" into "False", all worked fine, and I ran about 100 times with no issues which is great.

    I also considered using the Windows SDK (CreateFile), this works flawlessly with or without a background thread.

    Rudedog2: I am aware of the I/O taking a long time, but comparing SYSTEM.IO.DELETE with the Windows SDK's DELETE, it's a lot different, I think you are slightly missing the point, regardless of the I/O taking time, when calling the SYSTEM.IO.DELETE to use this I/O, it should wait for that I/O operation to complete, which can take seconds, or days, I don't expect (as a developer) that doing a DELETE that takes (say; an hour) to finish, for the command to execute in half-a-second telling me (ok, deleted) when the file actually has not been removed from the FS yet. and I think this is the problem.

    Anyhow, I fixed this in 3 ways:

    1) Either using the ugly Sleep/Loop above. Or;

    2) Replacing the SYSTEM.IO.FILE with the internal (non-wrapper) Windows File I/O SDK. Or:

    3) Best so far: Changing the thread from executing in the Background to a Foreground mode. This so far, does the trick, the price to pay here is CPU has to pay more attention, which I think is ok for today's systems anyway.

     

    I hope you (or all) found this useful, I reported this to Microsoft already, it's being investigated, and hopefully, a fix might be out at some point, who knows!

     

    Kind Regards

    Heider

     

     


    Heider Sati
    • Marked as answer by Heider Saturday, August 20, 2011 10:21 PM
    Saturday, August 20, 2011 10:21 PM