locked
Windows App Cert Kit will not fail a known bad Metro app

    Question

  • The Windows App Cert Kit will not fail a Metro app that I expect to fail, and does fail on other people's systems.

    1. Fresh install of Consumer Preview (VMWare image).

    2. Installed Visual Studio 11 Beta Ultimate

    3. Built, deployed, and ran applications (both a C++ and C# app)

    4. Run Windows App Cert Kit and it always returns a overall Pass with no detailed report.

    Here's what I've tried to get the apps to fail  to no avail (I tried the RegOpenKeyEx and debug both with and without the exception):

    Included a button that calls RegOpenKeyEx (C++ app from linked static library, C# from DllImport); deployed the Debug build; the main page's OnNavigatedTo throws an exception.

    I have seen this app fail the Windows App Cert Kit on other's systems and no one else seems to have the same problem. I can't figure out what is different about my system that causes this. Any one have a similar experience or any ideas?

    Friday, March 9, 2012 12:10 AM

All replies

  • Please do the following:

    1. Navigate to %userprofile%\AppData\Local\Microsoft\AppCertKit

    2. Post the contents of post_process_trace_SupportedAPIs_XXXXXX_XXXXXX.txt in a thread.

    -Raymond (MSFT)


    Raymond Parsons Microsoft Corporation This posting is provided "AS IS" with no warranties and confers no rights.

    Friday, March 9, 2012 2:38 AM
  • That file doesn't exist. I only have the files listed below in that directory.

     Directory of C:\Users\End User\AppData\Local\Microsoft\AppCertKit

    03/08/2012  03:54 PM             4,847 aitoutputToUpload_cbd7808e-c827-48c1-ba7f-4452d83ff848.xml
    03/08/2012  03:54 PM               749 aitoutputToUse_4eeeced1-fcd5-4b02-9bb8-49f5484c9533.xml
    03/08/2012  03:56 PM             4,792 appcertresults4.htm
    03/08/2012  03:56 PM             4,563 global_data.xml
    03/08/2012  03:56 PM            13,277 logotest_trace_382012_155244.txt
    03/08/2012  03:53 PM            37,276 programinventory_ba76ec5f-2738-4426-a812-c8857576b2e0.xml

    Friday, March 9, 2012 6:35 PM
  • If there are any errors in "logotest_trace_382012_155244.txt", can you please provide them here?


    Raymond Parsons Microsoft Corporation This posting is provided "AS IS" with no warranties and confers no rights.

    Friday, March 9, 2012 8:28 PM
  • I only see INFO lines (I ran it again since posting the files above so this result is from my latest run today 'logotest_trace_392012_132128.txt').

    Below are the lines related to the Supported API test since you asked about the trace file.

    appcertui.exe Information: 0 : INFO:Executing task SupportedApis.
    appcertui.exe Information: 0 : INFO:TAEF command-line "C:\Program Files\Windows Kits\8.0\App Certification Kit\Microsoft.Windows.SoftwareLogo.Tests.dll" /select:"@Name='Microsoft.Windows.SoftwareLogo.Tests.SupportedApis.SupportedApis.ExecuteTest'" /enableWttLogging /logfile:C:\Users\End User\AppData\Local\Microsoft\AppCertKit\Log_SupportedApis_VALIDATION.xml /p:DataFilePath=C:\Users\End User\AppData\Local\Microsoft\AppCertKit /p:TaskParameter=POST_PROCESS /p:PackageFullName=68b88582-0853-46da-9e3d-7fd5de896f84_1.0.0.0_x86__sft0jgsy4w8c2 /p:"AppxManifestFilePath=C:\Users\End User\Documents\Visual Studio 11\Projects\MetroTestWin32\Debug\MetroTestWin32\AppX\AppxManifest.xml".
    appcertui.exe Information: 0 : INFO:Task execution time: 00h:00m:01s.42ms
    appcertui.exe Information: 0 : INFO:Task execution result was success.

    Friday, March 9, 2012 9:30 PM
  • If possible, can you ZIP up all the log files in the 'AppCertKit' folder and share them on SkyDrive? Also, if you don't mind, can you share your APPX package(s) as well? While we may not need them (the APPX packages), they might turn out to be helpful for us during the investigation.


    Raymond Parsons Microsoft Corporation This posting is provided "AS IS" with no warranties and confers no rights.

    Saturday, March 10, 2012 1:00 AM
  • We tried to repro this internally and can confirm that the app that you provided failed the validation as expected.

    There seems to be something wrong with your local configuration. I'm especially concerned about the logs that you provided to us, as they seem to indicate that all of the tests ran correctly, but there are no XML/TXT files that would be created by each test.

    Are you 100% sure that you've provided logs from the proper AppCertKit folder? Can I suggest using Process Monitor from SysInternals to monitor the Cert Kit file system activity while you're running the validation and then verifying the folder that it's writing the logs to?

    If possible, can you try a similar setup on a different machine?


    Raymond Parsons Microsoft Corporation This posting is provided "AS IS" with no warranties and confers no rights.

    Monday, March 12, 2012 10:45 PM
  • I copied/pasted the folder path you gave me into explorer, so I'm as certain as I can be that it's the right location (and I turned Show Hidden on just in case). Also, the tool itself reports a Pass regardless of what folder I went to. That is, when I click the link to show results after the test completes it says everything Passed. I would expect the tool to know where to find the files even if I didn't. I also tried running it using appcert.exe but same thing happens. Maybe a permissions problem of some type?

    Could you clarify what you mean by "similar setup on a different machine"? I could try running my VM on a different machine, is that what you're after? There are many in the office that have a similar setup (install CP on a VM, install VS 11) and they don't seem to have any problems with the tool.

    I'll give Process Monitor a try. I should have some results in the next day or two.

    Tuesday, March 13, 2012 9:17 PM
  • I captured the registry, file system, and thread activity from the appcertui process with Process Monitor and put the log (PML formate) on SkyDrive. I wasn't sure what you would need so I only filtered by process name.

    https://skydrive.live.com/redir.aspx?cid=ff6e223a97b598a4&resid=FF6E223A97B598A4!141&parid=FF6E223A97B598A4!134&authkey=!ADj1SBjESl6jXek

    The path looked correct (C:\Users\End User\AppData\Local\Microsoft\AppCertKit), I didn't see other paths. I didn't really know what I was looking for so I couldn't debug it very well. I did notice some files it couldn't create (NAME NOT FOUND) that seemed that might be related. Each of the tests, prolog and epilog has a file it couldn't create with the name not found result. For example:

    5:37:08.7522532 PM appcertui.exe 3004 CreateFile C:\Users\End User\AppData\Local\Microsoft\AppCertKit\Log_LaunchAppCheck_VALIDATION.xml NAME NOT FOUND Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a

    I also noticed it might not have been able to create/find Microsoft.Windows.SoftwareLogo.Shared.XmlSerializers.exe, I saw many attempts to create the file but it couldn't find it and searching my C drive couldn't find it either. Is that file needed to create the test files?

    Examples:

    5:37:14.1772225 PM appcertui.exe 3004 CreateFile C:\Program Files\Windows Kits\8.0\App Certification Kit\Microsoft.Windows.SoftwareLogo.Shared.XmlSerializers\Microsoft.Windows.SoftwareLogo.Shared.XmlSerializers.exe PATH NOT FOUND Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a

    5:37:14.2092256 PM appcertui.exe 3004 CreateFile C:\Windows\assembly\GAC_64\Microsoft.Windows.SoftwareLogo.Shared.XmlSerializers NAME NOT FOUND Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a

    Thursday, March 15, 2012 1:00 AM
  • Hello,

    can you email me at mikewong at microsoft.com so we possibly do a live troubleshooting session on this issue?

    much appreciated,

    Mike Wong

    Microsoft Developer Support

    Tuesday, March 20, 2012 9:31 PM
    Moderator
  • Mike and I determine root cause and a defect has been submitted.

    The isssue was a space in my account name ("End User") and missing quotes on the command lines used to run the tests. In particular, the the /logfile and /p:DataFilePath arguments didn't quote the path, so the paths were not found and no results were saved even though the test ran and failed.


    Examples of defective command line arguments:
    /logfile:C:\Users\End User\AppData\Local\Microsoft\AppCertKit\Log_PerformanceLaunch_VALIDATION.xml
    /p:DataFilePath=C:\Users\End User\AppData\Local\Microsoft\AppCertKit


    Results were successfully reported (i.e., it showed the fails and the results files were in the AppCertKit folder) by using a new account with the name "NoSpace" on my system and also by modifying one of the tests to add quotes to the paths.

    Examples of good command line arguments:
    /logfile:"C:\Users\End User\AppData\Local\Microsoft\AppCertKit\Log_PerformanceLaunch_VALIDATION.xml"
    /p:"DataFilePath=C:\Users\End User\AppData\Local\Microsoft\AppCertKit"

    Thursday, March 22, 2012 12:39 AM
  • Thanks again Storm11 for taking the time to troubleshoot this one with me.

    much appreciated,

    mike

    Thursday, March 22, 2012 4:16 PM
    Moderator