locked
Consistent, fair benchmarks with Windows Prefetcher RRS feed

  • Question

  • I am trying to benchmark an application on Windows XP SP3, but I am having trouble with prefetcher's secret sauce.

    The best articles I found:
    1. http://en.wikipedia.org/wiki/Prefetcher
    2. http://www.microsoft.com/whdc/archive/benchmark.mspx
    3. http://blogs.msdn.com/ryanmy/archive/2005/05/25/421882.aspx

    My understanding is this
    1. Application prefetching works for basically all .exe executables
    2. Windows engages application prefetching at the time the application executable is started (in contrast to quickstarters like  "Fast Start", OSA, Adobe Quick Launch, etc.)

    My questions are
    1. Is my understanding (above) correct?
    2. Is there a diagnostic/debug log I can viewer to see what prefetcher is doing?
    3. Is prefetching analysis cumulative? To my surprise (below), I notice progressive improvements between reboots.

    Starting with a clean XP install, my benchmark does this
    1. Run the application twice (as suggested by article #2 above).  These times are not recorded.
    2. Runs advapi32.dll ProcessIdleTasks
    (as suggested by article #2 above)
    3. Reboot
    4. Run the benchmark 10 times.  The first of these 10 is a cold start.  The other 9 are warm.
    5. Reboot
    6. Repeat steps 4-5 another 15 times

    Here are my results starting with step #4.  The yellow parts concern me. 
    http://spreadsheets.google.com/pub?key=pgHmoZGs3VSt1VAozuqjPGA

    In conclusion, (1) what is going on? and (2)  how to do I get consistent results and a fair benchmark?  For consistency, instead of disregarding the first 2 iterations (as I do now), it seems I should disregard the first ~30?  However, ~30 is no longer a fair benchmark.


    Andrew
    Sunday, June 15, 2008 4:18 PM

Answers

  • I believe your understanding is correct.

     

    I don't know if there is a diagnostic / debug log, but the prefetcher effect is cumulative and would explain your results.

     

    My understanding is that you should turn off the prefetcher for performance testing (and repeatability). Prefetch is only going to make your app faster when it is ultimately turned on.

     

    Wednesday, June 25, 2008 10:54 PM