none
Powershell: HCK PackageWriter not always writing packages RRS feed

  • Question

  • Hello.  I have a PowerShell implementation of HCK automated testing, and I am having trouble with creating submission packages -- in one scenario it works great, and in another it does not.

    My HCK script does the following:

    1.  creates a project and machine pool

    2.  places the desired machine into that pool

    3.  runs the desired tests

    4.  creates the submission package

    5.  disposes the PackageWriter object

    6.  deletes the project and machine pool

    7.  places the machine back into the default pool

    8.  Result is a submission package and a machine ready for another test run.

     -- Perfect : )

    I am using another script to simultaneously call multiple instances of my HCK script, so that we can do the same for many machines.  It works except that only 3 or 4 out of 6 total packages get created.

    It appears there may be a race condition where some packages are created and then something gets clobbered.  The solution, I hoped, was to query the PackageWriter to see when it is finished, like this: 

    if ($PackageWriter.status -like "busy") {"wait till it is finished, don't clean up yet"}


    It seems there is not a property like that, but the C# example has implemented a function call that looks like this:

    packageWriter.SetProgressActionHandler(SubmissionCreationProgressHandler);

    and there is a function defined that looks like this:

    public static void SubmissionCreationProgressHandler(PackageProgressInfo info)
    {
        Console.WriteLine("Package progress {0} of {1} : {2}", info.Current, info.Maximum, info.Message);
    }

    So my two questions are, 1.  May I please request an example of how to use the SetProgressActionHandler in PowerShell (I've tried lots of internet searching and experimentation).  2.  The goal is to have all packages safely created before cleaning up -- can the ProgressActionHandler help with this?

    I would be happy to provide more information, of course.  But to keep this question politely concise, I tried to include what seemed most important.  Thank you!





    • Edited by hh-hh-hh Wednesday, November 27, 2013 4:19 PM
    Wednesday, November 27, 2013 1:50 AM

Answers

  • I'm glad you were able to get your system working.

    It is true that packaging is not setup for a multi-threaded environment primarily due to the common naming used for packageinfo.xml. I've noted this for future considerations.


    John -- This posting is provided "AS IS" with no warranties, and confers no rights.

    • Marked as answer by hh-hh-hh Monday, December 2, 2013 10:44 PM
    Monday, December 2, 2013 10:09 PM
  • John,

    Thank you for your help with this.  Initially I want to say that the program runs the script with multiple sessions, asynchronously.  To explain:  

    I have a python script that gathers information needed from our environment, and then calls the HCK.ps1 automation script and feeds it the needed parameters (test machine, controller, what tests to run, whether to create a package, and whether to "clean up" after a run).

    My python script is called by another ("Master") python script which uses threading to run HCK automation on one or more test machines.  So, if for example, we have a need to run HCK on six test machines, when it is running the Windows Task Manager (on the HCK controller) will show six 'powershell.exe *32' and six 'python.exe *32' Images.  The "Master" python script actually puts in a random sleep timer to stagger the 6 runs apart by a range of 1-10 seconds to help make all six runs start properly (without it, not all 6 runs start because one or more machines do not initialize in the HCK).  That's another issue I need to solve, but the staggered sleep timer is a (hack) workaround to get us going for now.

    At package creation time, I thought that if we are able to get the runs to start, then we should see all 6 packages at the end.  Nope.  However, I did hack another workaround by placing the packagewriter.save() call into a try-catch within a while loop.  I apologize that it is a hack, and a mere incomplete one at that (unfinished and amateur), but the first experiment that gave all 6 packages was:

    $exitPackageCreation = $false
    while ($exitPackageCreation -eq $false)
    {
       try
       {
          $packageWriter.Save($saveFileName)
          Write-Host "`nSuccessfully created package for $projectName`n"
          $exitPackageCreation = $true
       }
       catch
       {
          Write-Host "`nPackageInfo.xml was inaccessible, could not create package for $projectName.`n"
          Start-Sleep -s 5
       }
    }
    With that in place, all the packages were created.  As that is only a workaround, I believe my problem is that multiple instances of PackageWriter wants to access this PackageInfo.xml and they just can't.  Perhaps if there was a way to give a custom name to the PackageInfo.xml, or if that file could have a unique ID for each 'thread', it might resolve the conflict.  However, perhaps something I ought to do on my side would help, as illustrated by the presence of a workaround to kick off more than one job and have all of them successfully initialize their test machines. 


    • Edited by hh-hh-hh Monday, December 2, 2013 9:41 PM
    • Marked as answer by hh-hh-hh Monday, December 2, 2013 10:44 PM
    Monday, December 2, 2013 9:39 PM

All replies

  • I'm not certain that the progress handler would necessarily help with determining completion because on return from PackageWriter.Save(...), the package should be complete.

    The value of using the progress handler would be to see the steps taken when creating the package successfully and see what steps were missed when the package was not created.

    That said, I'm interested in knowing if the Windows Event logs show anything (i.e. failures). See if the event log - Event Viewer | Applications and Services Logs | HCK - Hardware Certification Kit - provides any insight.


    John -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Wednesday, November 27, 2013 4:31 PM
  • John, 

    Thank you for your reply.  I looked at the log you specified and also carefully checked my console output.  I did find something interesting in the console output.  To have fresh information, I ran my script again, which ran the DevFund INF Test on six machines.  Three of the runs resulted in packages being created, and three did not.

    From the console output, I see an error message that was generated three times, each was with the machine that did not result in a submission package (I'm using a certain infrastructure to launch my scripts each time we have a new build to test):

    10:21:32 Exception calling "Save" with "1" argument(s): "The process cannot access the 
    10:21:32 file 'C:\Windows\TEMP\PackageInfo.xml' because it is being used by another 
    10:21:32 process."At C:\jenkins\workspace\winCertification_forTesting\HCK
    10:21:32 Controller\myController\win\hck\hck.ps1:1012 char:5
    10:21:32 +     $packageWriter.Save($saveFileName);
    10:21:32 +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    10:21:32     + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    10:21:32     + FullyQualifiedErrorId : IOException


    However, the other three machines did not have this error message, and for each of them, there is a submission package written to C:\.

    The Windows Application logs report the following messages (I did not include the sources 'Target' and 'Test' as those messages appear normal -- also I'm not sure if the messages from LogoManager would be consequential, as it provided an error for all 6 machines, but 3 of those machines resulted in good packages):

    Level	Date and Time	Source	Event ID	Task Category
    Information	11/27/2013 10:21:04 AM	Package	0	None	LogoPackage: Created temp file 'C:\Windows\TEMP\tmp30DD.tmp' for package.
    Error	11/27/2013 10:21:04 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Information	11/27/2013 10:21:04 AM	Package	0	None	LogoPackage: Created temp file 'C:\Windows\TEMP\tmp2F78.tmp' for package.
    Information	11/27/2013 10:21:04 AM	Package	0	None	LogoPackage: Created temp file 'C:\Windows\TEMP\tmp2F77.tmp' for package.
    Information	11/27/2013 10:21:04 AM	Package	0	None	LogoPackage: Created temp file 'C:\Windows\TEMP\tmp2F76.tmp' for package.
    Information	11/27/2013 10:21:04 AM	Package	0	None	LogoPackage: Created temp file 'C:\Windows\TEMP\tmp2F67.tmp' for package.
    Information	11/27/2013 10:21:04 AM	Package	0	None	LogoPackage: Created temp file 'C:\Windows\TEMP\tmp2F66.tmp' for package.
    Error	11/27/2013 10:21:04 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:21:04 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:21:04 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:21:04 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:21:04 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:20:42 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:20:40 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:20:39 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:20:39 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:20:39 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    Error	11/27/2013 10:20:39 AM	LogoManager	0	None	ConnectToDataStore: server is empty
    

    I noticed that the PackageInfo.xml is currently updated with info for one of the three machines that was able to produce a submission package.

    I ran it again and noticed on the next run that only 2 of the 6 packages were successfully created.  It changes from run to run.  I also tried enabling and disabling the PackageWriter.Dispose() call, as well as my logic to clean up the projects and machine pools after (tried all 4 combinations of these settings).

    Also noticed that the \TEMP\tmp*.tmp packages that are created were automatically deleted later on, but not always.  On one run, all were deleted.  On the next run, three of six were deleted, and of the three remaining, one matched a successfully created package, and the other two were for machines that did not have a package.

    As there appears to be only one PackageInfo.xml file, and the console out shows errors because it is being used by another process, I wondered if there is a way to create a PackageInfo.xml for each process, perhaps with a unique name?  Also wondered (not sure if it would help) to attempt to create the PackageWriter object with a unique name for each of the six instances.  Or is there a way to have each process politely wait its turn to use the PackageInfo.xml file?

    Wednesday, November 27, 2013 6:54 PM
  • Thank you for the detailed reply. While I don't have a specific answer to your issue, I am familiar with the PackageInfo.xml file and will look into what might be holding the context of the file for subsequent runs.

    When you run your program that calls your script multiple times, are these synchronous or asynchronous (i.e. run on different threads)?


    John -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Monday, December 2, 2013 6:10 PM
  • John,

    Thank you for your help with this.  Initially I want to say that the program runs the script with multiple sessions, asynchronously.  To explain:  

    I have a python script that gathers information needed from our environment, and then calls the HCK.ps1 automation script and feeds it the needed parameters (test machine, controller, what tests to run, whether to create a package, and whether to "clean up" after a run).

    My python script is called by another ("Master") python script which uses threading to run HCK automation on one or more test machines.  So, if for example, we have a need to run HCK on six test machines, when it is running the Windows Task Manager (on the HCK controller) will show six 'powershell.exe *32' and six 'python.exe *32' Images.  The "Master" python script actually puts in a random sleep timer to stagger the 6 runs apart by a range of 1-10 seconds to help make all six runs start properly (without it, not all 6 runs start because one or more machines do not initialize in the HCK).  That's another issue I need to solve, but the staggered sleep timer is a (hack) workaround to get us going for now.

    At package creation time, I thought that if we are able to get the runs to start, then we should see all 6 packages at the end.  Nope.  However, I did hack another workaround by placing the packagewriter.save() call into a try-catch within a while loop.  I apologize that it is a hack, and a mere incomplete one at that (unfinished and amateur), but the first experiment that gave all 6 packages was:

    $exitPackageCreation = $false
    while ($exitPackageCreation -eq $false)
    {
       try
       {
          $packageWriter.Save($saveFileName)
          Write-Host "`nSuccessfully created package for $projectName`n"
          $exitPackageCreation = $true
       }
       catch
       {
          Write-Host "`nPackageInfo.xml was inaccessible, could not create package for $projectName.`n"
          Start-Sleep -s 5
       }
    }
    With that in place, all the packages were created.  As that is only a workaround, I believe my problem is that multiple instances of PackageWriter wants to access this PackageInfo.xml and they just can't.  Perhaps if there was a way to give a custom name to the PackageInfo.xml, or if that file could have a unique ID for each 'thread', it might resolve the conflict.  However, perhaps something I ought to do on my side would help, as illustrated by the presence of a workaround to kick off more than one job and have all of them successfully initialize their test machines. 


    • Edited by hh-hh-hh Monday, December 2, 2013 9:41 PM
    • Marked as answer by hh-hh-hh Monday, December 2, 2013 10:44 PM
    Monday, December 2, 2013 9:39 PM
  • I'm glad you were able to get your system working.

    It is true that packaging is not setup for a multi-threaded environment primarily due to the common naming used for packageinfo.xml. I've noted this for future considerations.


    John -- This posting is provided "AS IS" with no warranties, and confers no rights.

    • Marked as answer by hh-hh-hh Monday, December 2, 2013 10:44 PM
    Monday, December 2, 2013 10:09 PM
  • John,

    Thank you.  Much appreciated : )

    I'll put a few more suggestions into the hat:

    1.  Let PackageWriter have a status property that we can query, and if busy, wait before calling it.  Or, better, have PackageWriter automatically (and magically) handle all the multiple requests handed to it.

    2.  Could use more sample PowerShell code on multi-threaded HCK testing, from start to finish, with submission packages as the end result.

    3.  It would be wonderful if the HCK API reference included more human touch -- Show what a class contains, but also show the parent class, and all the child classes.  Provide examples of how to learn what objects, properties, classes, and methods are available to the PowerShell HCK Automation coder, and how to see what's available using IntelliSense while coding in the PowerShell ISE.  Probably, most helpful would be to know what objects you must instantiate before you can use a property or method from another object.  For example, you must have a $RootMachinePool ($Manager.GetRootMachinePool) before you can access one of the child pools.  Before you have a $RootMachinePool, I believe you must first have a $Manager (DatabaseProjectManager).  If there's a more direct route to that, show that in the documentation.

    (Will admit that the object hierarchy is a good place to start: http://msdn.microsoft.com/en-us/library/windows/hardware/jj124757.aspx); would like to see all the objects.

    Thanks again for all the help : )   Sure is nice!



    • Edited by hh-hh-hh Wednesday, December 4, 2013 10:08 PM
    Monday, December 2, 2013 10:37 PM