none
"Flush" vs. "Tableupdate(1,2,.T.)" ?

    Question

  • The question is: When is it wiser to "flush force" (data to disk) vs. =Tableupdate(.T.), =Tableupdate(1), etc. ?  What is the usual paradigm?  Can I forget flush?

    Is flush slow in the LAN setting(s)?  Does flush hog resources?

    Does frequent-flush help prevent corrupting tables during intense LAN data-manipulation, crashes, and/or power outs

    Is is dangerous to rely too much on Tableupdate() alone when tables stay open all day ?

    As always, thanks in advance for any thoughts?

    Thursday, February 08, 2007 3:50 AM

Answers

  • I believe FLUSH renders VFP's buffer mechanism useless and results to slower performance. IMO, I'd stick to buffering with my applications and use FLUSH only when I get a lot of data corruption, which hasn't happened to me yet.
    Thursday, February 08, 2007 5:45 AM
  • >> The question is: When is it wiser to "flush force" (data to disk) vs. =Tableupdate(.T.), =Tableupdate(1), etc. ? 

    The answer is NEVER! TableUpdate() and Flush have nothing to do with each other. You need TableUpdate() to commit pending changes in a table buffer to the underlying data file. Flush does NOT do this!!! Run the following program:

    CLOSE ALL
    IF FILE
    ( 'testit.dbf' )
     
    DELETE FILE testit.*
    ENDIF
    CREATE TABLE
    testit ( F1 C(10), F2 C(10))
    CURSORSETPROP( 'Buffering', 5 )

    *** Use Flush ***
    INSERT INTO testit VALUES ("Row 1", "Value 1" )
    INSERT INTO testit VALUES ("Row 2", "Value 2" )
    FLUSH
    ? "After Flush: ", RECCOUNT()
    TABLEREVERT( .T. )
    ?
    "After TableRevert: ", RECCOUNT()
    ?
    *** Use TableUpdate ***
    INSERT INTO testit VALUES ("Row 1", "Value 1" )
    INSERT INTO testit VALUES ("Row 2", "Value 2" )
    IF TABLEUPDATE( .T., .T. )
      ?
    "After TableUpdate: ", RECCOUNT()
    ENDIF
    TABLEREVERT
    ( .T. )
    ?
    "After TableRevert: ", RECCOUNT()

    As you can see, FLUSH clearly does not commit pending changes. The whole purpose of FLUSH is to force the operating system to clear its buffers, not VFP!

    >> What is the usual paradigm? 

    You should only use flush to ensure that the operating system clears any cached data - but this has always been a very flakey area in Windows (the caching system is really intended for files and like so many of Microsoft's tools does not handle data well) - so in VFP 9.0 the keyword "FORCE" was added to VFP.  This now calls the windows API to force Windows to clear any cached data and write it immediately to disk. 

    (It is worth noting that one the first things that SQL Server does when it is installed is to disable Windows caching!)

    >> Can I forget flush?

    Yes, unless you have a specific problem with data not being cleared from the Operating System's cache.

     

     

     

    Thursday, February 08, 2007 10:38 AM
  • >> An updated open data-file remains unsaved and not "Modified" (according to fdate("myData.dbf") and Windows Explorer "Modified")

    Thae fact of the matter is that I have been working in VFP for more than 20 years now, and in all that time I have NEVER needed to use FLUSH. Now whether this iis a consequence of the fact that Windows now enables write behind caching on its Operating Systems by default (the first thing I do on any windows box is DISABLE it!) or because my applications don't rely on real-time updates between users I don't know. 

    My first question in the scenario you describe would be "Do you know for sure that the update succeeded?" and the second would be "How do you know that the data is NOT being retrieved upon demand?" I am not an expert on write-behind caching by any means, but my understanding is that the way it works is that Windows checks the cache FIRST, if it finds the requested data it returns it from there instead of accessing the file on disk. When the system is idle, it updates the file on disk and clears the cache. So the low-level test of the file on disk would not necessarily mean that your data is "lost" - merely that it is still in the Cache.

    >> But your keen observation of de-chaching in SQL-Server seems to validate my suspicions that data should be force-flushed oft-betimes VFP9, perhaps during each and every save (tableupdate())  I think Budoi, that you initially understood #1 below.

    Better to turn off write-behind caching on your data server - that obviates the need for this (and is presumably why SQL Server does it)

    >>Flush Force to be used in conjunction with Tableupdate() during every Save as:

    This smacks of using a sledgehammer to crack a walnut. As I said, I have never found this to be necessary and while I know of some applications where it is necessary to have immediate disk writes (inventory management systems running over a WAN for example) they are few and far between. But of course, if you afraid that your application is running too fast, and want to slow it down "prophylactically" go right ahead.

    Friday, February 09, 2007 10:51 AM

All replies

  • I believe FLUSH renders VFP's buffer mechanism useless and results to slower performance. IMO, I'd stick to buffering with my applications and use FLUSH only when I get a lot of data corruption, which hasn't happened to me yet.
    Thursday, February 08, 2007 5:45 AM
  • >> The question is: When is it wiser to "flush force" (data to disk) vs. =Tableupdate(.T.), =Tableupdate(1), etc. ? 

    The answer is NEVER! TableUpdate() and Flush have nothing to do with each other. You need TableUpdate() to commit pending changes in a table buffer to the underlying data file. Flush does NOT do this!!! Run the following program:

    CLOSE ALL
    IF FILE
    ( 'testit.dbf' )
     
    DELETE FILE testit.*
    ENDIF
    CREATE TABLE
    testit ( F1 C(10), F2 C(10))
    CURSORSETPROP( 'Buffering', 5 )

    *** Use Flush ***
    INSERT INTO testit VALUES ("Row 1", "Value 1" )
    INSERT INTO testit VALUES ("Row 2", "Value 2" )
    FLUSH
    ? "After Flush: ", RECCOUNT()
    TABLEREVERT( .T. )
    ?
    "After TableRevert: ", RECCOUNT()
    ?
    *** Use TableUpdate ***
    INSERT INTO testit VALUES ("Row 1", "Value 1" )
    INSERT INTO testit VALUES ("Row 2", "Value 2" )
    IF TABLEUPDATE( .T., .T. )
      ?
    "After TableUpdate: ", RECCOUNT()
    ENDIF
    TABLEREVERT
    ( .T. )
    ?
    "After TableRevert: ", RECCOUNT()

    As you can see, FLUSH clearly does not commit pending changes. The whole purpose of FLUSH is to force the operating system to clear its buffers, not VFP!

    >> What is the usual paradigm? 

    You should only use flush to ensure that the operating system clears any cached data - but this has always been a very flakey area in Windows (the caching system is really intended for files and like so many of Microsoft's tools does not handle data well) - so in VFP 9.0 the keyword "FORCE" was added to VFP.  This now calls the windows API to force Windows to clear any cached data and write it immediately to disk. 

    (It is worth noting that one the first things that SQL Server does when it is installed is to disable Windows caching!)

    >> Can I forget flush?

    Yes, unless you have a specific problem with data not being cleared from the Operating System's cache.

     

     

     

    Thursday, February 08, 2007 10:38 AM
  • >> I believe FLUSH renders VFP's buffer mechanism useless and results to slower performance

    No, this is totally incorrect. See my reply below!

    FLUSH is concerned only with the operating system buffers, not VFP's native table buffers!

    Thursday, February 08, 2007 10:40 AM
  •  AndyKr wrote:

    >> I believe FLUSH renders VFP's buffer mechanism useless and results to slower performance

    No, this is totally incorrect. See my reply below!

    FLUSH is concerned only with the operating system buffers, not VFP's native table buffers!

    Sorry Andy, I stand corrected. Philip was asking about FLUSH vs. TABLEUPDATE and I thought that if he used the former he wouldn't be using the latter anymore.

    Friday, February 09, 2007 12:25 AM
  • Budoi and Andy,

    While issuing FDATE() to display "Last Modified", I *discovered* open tables don't write to disk after commanding "tableupdate(.T.)"  An updated open data-file remains unsaved and not "Modified" (according to fdate("myData.dbf") and Windows Explorer "Modified")

    Thank you for clarifying "flush force" and its remote (OS) applicability.  Andy, your thoughtful code-example clears things up alot.  But your keen observation of de-chaching in SQL-Server seems to validate my suspicions that data should be force-flushed oft-betimes VFP9, perhaps during each and every save (tableupdate())  I think Budoi, that you initially understood #1 below.

    Flush Force to be used in conjunction with Tableupdate() during every Save as:

    1) A prophylactic mechanism to help protect Server-Data-Files from becoming corrupted during their extensive habitation in RAM. 

    2) A better time-marker when a data file is updated, for: "Saved" = "Last-Modified"

    Andy, Budoi, and/or others, does this not seem more logical for the user?

    Friday, February 09, 2007 3:22 AM
  • >> An updated open data-file remains unsaved and not "Modified" (according to fdate("myData.dbf") and Windows Explorer "Modified")

    Thae fact of the matter is that I have been working in VFP for more than 20 years now, and in all that time I have NEVER needed to use FLUSH. Now whether this iis a consequence of the fact that Windows now enables write behind caching on its Operating Systems by default (the first thing I do on any windows box is DISABLE it!) or because my applications don't rely on real-time updates between users I don't know. 

    My first question in the scenario you describe would be "Do you know for sure that the update succeeded?" and the second would be "How do you know that the data is NOT being retrieved upon demand?" I am not an expert on write-behind caching by any means, but my understanding is that the way it works is that Windows checks the cache FIRST, if it finds the requested data it returns it from there instead of accessing the file on disk. When the system is idle, it updates the file on disk and clears the cache. So the low-level test of the file on disk would not necessarily mean that your data is "lost" - merely that it is still in the Cache.

    >> But your keen observation of de-chaching in SQL-Server seems to validate my suspicions that data should be force-flushed oft-betimes VFP9, perhaps during each and every save (tableupdate())  I think Budoi, that you initially understood #1 below.

    Better to turn off write-behind caching on your data server - that obviates the need for this (and is presumably why SQL Server does it)

    >>Flush Force to be used in conjunction with Tableupdate() during every Save as:

    This smacks of using a sledgehammer to crack a walnut. As I said, I have never found this to be necessary and while I know of some applications where it is necessary to have immediate disk writes (inventory management systems running over a WAN for example) they are few and far between. But of course, if you afraid that your application is running too fast, and want to slow it down "prophylactically" go right ahead.

    Friday, February 09, 2007 10:51 AM
  • Thank you very much, Andy.  This helps immensely.

    Well, I rule out the first hypothesis of "data-safety" based on your strong logic.  We won't implement "flush force" in our EMR-apps.  I stand corrected.  Also I am humbled and apologize for my severe ignorance of SQL.

    Most table-corruption we've experienced probably occurred during hasty file transfers, backups, and/or recoveries (in the last 10-12 years).  Also our table-repair routines have seemed 'apt' enough to repair corrupted records.

    My 2nd hypothesis: "Flush force" to force VFP-apps to behave themselves as "Modified" in Explorer (and fdate("myTable.dbf")) ...

    ... I conclude that would also be a "sledgehammer to crack a walnut".  Instead I'll test DATETIME() expression-markers conjoined with Tableupdate() in our SAVE methods.

    Friday, February 09, 2007 6:11 PM