locked
.net file random access recoard locking RRS feed

  • Question

  • Summary:

    In .net When locking a record in a random access file we cannot access records ahead of the locked record in the file.
    To demonstrate the issue I have written two simple programs one opens and locks a record and the other tries to read through.
    The results are that when locking record number 9 out of the 10 in the first program we are able to read records 1 and 2 but no more! The expectation (And this is our experience with VB6) is that you should be able to read all the records apart from the one you have locked.
    Has anyone seen this problem? Am I doing something strange? Any work around?

    Demo Code:
    Program 1 Create/Open/Lock

    Structure Person
        <VBFixedString(600)> Dim name As String
    End Structure
    Public People As Person

    Sub Main()

    Dim FileName As String = "test.a" Dim ListofName() As String = {"Name1", "Name2", "Name3", "Name4", "Name5", "Name6", "Name7", "Name8", "Name9", "Name10"} Try Dim FileNumber1 As Integer = FreeFile() FileOpen(FileNumber1, FileName, OpenMode.Random, OpenAccess.ReadWrite, OpenShare.Shared, 600) FileGet(FileNumber1, People, 1) 'Create File if needs be If People.Name = "" Then For A = 1 To 10 People.Name = ListofName(A - 1) FilePut(FileNumber1, People, A) Next End If 'Lock the recoard we want for testing Lock(FileNumber1, 9) Catch ex As Exception FileClose() End Try FileClose() End Sub

    Program 2 Open and try and read

    Sub Main()
    
        Dim FileName As String = "C:\**Location of first program file**\test.a"
    
        Try
    
            Dim FileNumber1 As Integer = FreeFile()
            FileOpen(FileNumber1, FileName, OpenMode.Random,
                     OpenAccess.ReadWrite, OpenShare.Shared, 600)
    
            FileGet(FileNumber1, People, 2)
    
            'See  how much of the file we can read
            For A = 1 To 10
                FileGet(FileNumber1, People, A)
                System.Diagnostics.Debug.WriteLine(People.Name.ToString)
            Next
    
        Catch ex As Exception
            FileClose()
        End Try
        FileClose()
    End Sub

    Edit 0.1: We have found that the deeper an individual record is locked within the file the more bytes/records are inaccessible before the locked one.

    Thanks,

    Richard.


    • Edited by Rboydo Thursday, July 20, 2017 11:30 PM edit
    Thursday, July 20, 2017 11:25 PM

Answers

  • In my opinion the problem is not caused by incorrect locking, but by the buffered nature of reading operations. For performance reasons, the readers usually read more bytes keeping them in some cache.  In this case they could reach a neighbour locked region.

    As a workaround, try reading only the required amount of bytes. A possible solution:

    <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Ansi, Pack:=1)>
    Structure Person
       <MarshalAs(UnmanagedType.ByValTStr, SizeConst:=600)>
       <VBFixedString(600)>
       Dim name As String
    End Structure
    
    . . .
    
    Using fs = New FileStream(FileName, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite, Marshal.SizeOf(People))
       For A = 1 To 10
          Try
             Console.WriteLine("Trying " & A & "...")
    
             Dim b() As Byte
             ReDim b(Marshal.SizeOf(People) - 1)
             fs.Seek((A - 1) * Marshal.SizeOf(People), SeekOrigin.Begin)
             fs.Read(b, 0, b.Length)
             Dim h = GCHandle.Alloc(b, GCHandleType.Pinned)
             People = Marshal.PtrToStructure(Of Person)(h.AddrOfPinnedObject())
             h.Free()
    Console.WriteLine(People.name.Trim()) Catch ex As Exception Console.WriteLine("ERROR " & A & " " & ex.Message) End Try Next End Using





    Friday, July 21, 2017 11:26 AM
  • In my opinion the problem is not caused by incorrect locking, but by the buffered nature of reading operations. For performance reasons, the readers usually read more bytes keeping them in some cache.  In this case they could reach a neighbour locked region.

    Hi Viorel,

    You are both a scholar and a gentleman! I have studied your code to understand it better and just tested it again and found that if I lock any one of the 10 records in question using my code example I am able to then read each part of the file apart from the record which is locked using your code, which is the expected result!! :)

    So the conclusion is this that with .net if you use FileGet or (e.g. System.IO.BinaryReader.ReadDouble()) the reader not only reads what you want but also buffers content meaning that a locked record ahead within the file is hit causing read failure.

    But when using System.IO.FileStream.Read() and specifying exactly the number of bytes you want the buffer is not created allowing you to read all the records in a file except the locked one.

    Thank you very much I hope you have a good Friday!!

    Richard.


    • Marked as answer by Rboydo Friday, July 28, 2017 11:32 AM
    • Edited by Rboydo Saturday, July 29, 2017 10:01 PM correction
    Friday, July 28, 2017 11:28 AM

All replies

  • Any work around?

    Stop using Access and use another DB like SQLite of MS SQL Server Express.

    Friday, July 21, 2017 3:34 AM
  • Stop using Access and use another DB like SQLite of MS SQL Server Express.

    Access is not involved.  Perhaps you mean 'Stop using VB6-compatible file management methods and use the .Net methods instead'. 

    Friday, July 21, 2017 4:26 AM
  • The results are that when locking record number 9 out of the 10 in the first program we are able to read records 1 and 2 but no more! The expectation (And this is our experience with VB6) is that you should be able to read all the records apart from the one you have locked.
    Has anyone seen this problem? Am I doing something strange? Any work around?


    Hi Rboydo,

    I have test your code, and I first run then program 1 and I get test.a file, which is contests of ListofName, then I run the program2, I can Debug.writeline the evertthing in the test.a.

    Best Regards,

    Cherry



    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, July 21, 2017 5:35 AM
  • In my opinion the problem is not caused by incorrect locking, but by the buffered nature of reading operations. For performance reasons, the readers usually read more bytes keeping them in some cache.  In this case they could reach a neighbour locked region.

    As a workaround, try reading only the required amount of bytes. A possible solution:

    <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Ansi, Pack:=1)>
    Structure Person
       <MarshalAs(UnmanagedType.ByValTStr, SizeConst:=600)>
       <VBFixedString(600)>
       Dim name As String
    End Structure
    
    . . .
    
    Using fs = New FileStream(FileName, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite, Marshal.SizeOf(People))
       For A = 1 To 10
          Try
             Console.WriteLine("Trying " & A & "...")
    
             Dim b() As Byte
             ReDim b(Marshal.SizeOf(People) - 1)
             fs.Seek((A - 1) * Marshal.SizeOf(People), SeekOrigin.Begin)
             fs.Read(b, 0, b.Length)
             Dim h = GCHandle.Alloc(b, GCHandleType.Pinned)
             People = Marshal.PtrToStructure(Of Person)(h.AddrOfPinnedObject())
             h.Free()
    Console.WriteLine(People.name.Trim()) Catch ex As Exception Console.WriteLine("ERROR " & A & " " & ex.Message) End Try Next End Using





    Friday, July 21, 2017 11:26 AM
  • hi DA924x

    Thanks for your replay but, yeah not using Access its VB6 writing, locking and reading flatfiles

    Thanks, Richard.


    • Edited by Rboydo Wednesday, July 26, 2017 9:33 PM correction
    Wednesday, July 26, 2017 9:17 PM
  • Hi Acamar,

    Thanks for your reply, yeah we tested using .net with filestream and binaryreader and came up with the same issue. Fails to read past a seatrain number of bytes in a preceding records. The belief is that the ported over FileGet users filestream underneath.

    If you would like we can post .net test code to demonstrate the same issue.

    Thanks,

    Richard.



    • Edited by Rboydo Wednesday, July 26, 2017 10:34 PM correction
    Wednesday, July 26, 2017 9:22 PM
  • Hi Cheery, thanks for your reply.

    The idea of record locking is so other users using the same file system are able to read and modify a given record in a multi user environment.

    As such both programs have to be run at the same time.

    The first is run until the given record is locked and paused.

    Then if you start to run the second program and step through the reading/displaying loop you will get an error when trying to read record 3 saying 'part or all of the file is locked'

    Thanks, Richard.

    Wednesday, July 26, 2017 9:30 PM
  • Thanks for your reply, yeah we tested using .net with filestream and byte reading and came up with the same issue.

    Yes.  I wasn't suggesting that my revision of the original response was actually going to solve anything.

    Whenever I have done this I have implemented locking by maintaining a separate shared memory file with a lock register, and managed the locking by having each application checking in with the register before trying to access the data files. This can be done through some common code with a simple test/lock/unlock interface.  But that was a long time back - modern database management systems will look after record locking for you.

    Wednesday, July 26, 2017 9:35 PM
  • Rboydo,

    I almost cannot type it, but in a way DA is right. But then stop using Random Access files. It is from the time just following on the IBM 1401 when disks were introduced. 

    https://en.wikipedia.org/wiki/IBM_1401

    Use a database and for sure not recordlocking. In a little bit information store it gives alone dead locks. 

    We know already a long time that concurrency problems are much less than people thought about in the past. (However because of all the dead locks we were confrontated with it if it was huge)

    Therefore Optimistic Concurrency is used. The principle is easy. You read a record with for instance a TimeStamp (but it can also be all data). 

    Just before you writeback you test if the timestamp is the same (this should be done on the server) if that is the case, you write back the data and a new timestamp. 

    All Microsoft ORM (TableManagers, Linq to SQL and Linq to Enteties) have this build in the logic. 

     


    Success
    Cor



    Wednesday, July 26, 2017 10:16 PM
  • Hi Viorel, thank you for your reply.

    Good idea with the buffering its something we had though about too but we had tested it with filestream and binaryreader to try and confirm this but as you say the binaryreader could be reading ahead..

    Our normal file dose not have one fixed length string but around 30 to 40 variables of differing sizes and as we go though and try and read them it fails trying to read and populate a given variable in preceding records e.g. part way through the 3rd.

    I don't understand your code fully however I have tested it and it fails to even read the first record in the file returning the same error we are getting "The process cannot access the file because another process has locked a portion of the file." But the point with record locking is that you should be able to read all the other records ;) 

    Happy to put together a System.IO filestream and binaryreader demo if needed so others can see it failing part way through reading a record.

    Have also posted the same question over on stackoverflow and others had made some interesting points but no answer.

    stackoverflow

    Thanks Richard.




    • Edited by Rboydo Wednesday, July 26, 2017 11:23 PM
    Wednesday, July 26, 2017 10:33 PM
  • Whenever I have done this I have implemented locking by maintaining a separate shared memory file with a lock register, and managed the locking by having each application checking in with the register before trying to access the data files. This can be done through some common code with a simple test/lock/unlock interface.  But that was a long time back - modern database management systems will look after record locking for you.

    Agree in an ideal world we would not be using flat files SQL is soo much better at this out of the box.

    The hope was that as MS has stated on the MSDN documentation that it was possible to lock parts of a file that you would be able to read the other parts of the file reliability as you could with VB6.

    filestream.lock

    Developing our own lock registry was something I had thought about but the rework would be close to just moving over to SQL background. 

    Wednesday, July 26, 2017 11:48 PM
  • I almost cannot type it, but in a way DA is right. But then stop using Random Access files. It is from the time just following on the IBM 1401 when disks were introduced. 

    Yeah agree SQL would be a better way to do this but it would be a big job and the hope was that someone had had experience of getting around this issue.

    Wednesday, July 26, 2017 11:54 PM
  • In my opinion the problem is not caused by incorrect locking, but by the buffered nature of reading operations. For performance reasons, the readers usually read more bytes keeping them in some cache.  In this case they could reach a neighbour locked region.

    Hi Viorel,

    You are both a scholar and a gentleman! I have studied your code to understand it better and just tested it again and found that if I lock any one of the 10 records in question using my code example I am able to then read each part of the file apart from the record which is locked using your code, which is the expected result!! :)

    So the conclusion is this that with .net if you use FileGet or (e.g. System.IO.BinaryReader.ReadDouble()) the reader not only reads what you want but also buffers content meaning that a locked record ahead within the file is hit causing read failure.

    But when using System.IO.FileStream.Read() and specifying exactly the number of bytes you want the buffer is not created allowing you to read all the records in a file except the locked one.

    Thank you very much I hope you have a good Friday!!

    Richard.


    • Marked as answer by Rboydo Friday, July 28, 2017 11:32 AM
    • Edited by Rboydo Saturday, July 29, 2017 10:01 PM correction
    Friday, July 28, 2017 11:28 AM