none
Certification failed and I don't understand why

    Question

  • Hi,
    My application (written in C#) failed the certification tests.
    Security tests and Technical compliance were ok, but it failed in the Content compliance.

    I have attached (below) the details...

    I don't understand why it's failing...
    It seems saying that credentials for using it are missing, but it needs a Facebook account and EVERYONE have a fb account. Also, I'm not going either to give mine, or to create a fake profile.

    About the speed running, I don't know what's wrong... it's super faster opening and suspending (it does nothing).

    Thanks,
    Marco


    Content compliance:                     failed                     

          This test evaluates your software for content compliance with Windows Store Certification Requirements.       Learn more

    Your app doesn't meet requirement 1.2.                                                               Learn more                                                                 

          Questions to consider:    

    •           Does your app require a user account? If so, you need to include a test account in the Notes to Testers field.        
    •           Can users make purchases through the app? If so, you need to provide a way to test those purchases.        

          Commons reasons why apps fail this requirement:    

    •           The app includes non-functional sections or contains placeholders (labeled "coming soon", "more to come", "not available yet" and so on) for primary user scenarios.        
    •           The app doesn't work on all the architectures that it claims to support. For example, if you state that your app works on any CPU, it must work on all architectures, including ARM.        
    •           The app description is misleading or vague.        
    •           The app description uses screenshots or statements that imply features that don't appear to be implemented.        
    •           The app doesn't respond to "play" and "pause" keyboard events to allow users to control audio playback.        
    •           The app description doesn't explicitly state any hardware or network requirements.        
    Your app doesn't meet requirement 3.8.                                                               Learn more                                                                 

          The launch time and suspend time of an app depends on many variables, such as system load and configuration, which can vary from one computer to another and from one test to another. Because of this variability, a launch time or a suspend time that is just under the limit in one instance could be just over the limit, and fail the test, in another.      

          If the app's launch time or suspend time is close to the limit, review what the app is doing to handle those events and look for ways to reduce that activity.     

                                   G:: Missing or incomplete account information provided to reviewer. This prevented us from completing our review of this app. A:: This app failed performance test in the Windows App Certification Kit. The Performance Analyzer for HTML5 Apps enables you to find common performance issues in your app that prevent it from being fast and fluid. The tool tests your apps in a wide variety of scenarios including fast UI responsiveness, fluid graphics animations, activation time, suspend times, memory foot print, memory leaks, Idle efficiency, and touch responsiveness. See: http://msdn.microsoft.com/en-us/library/windows/apps/jj553524.aspx
    Wednesday, October 10, 2012 8:58 AM

Answers

  • So? What should we do?
    Resubmit the app, waiting for the certification team to fix server-side the submission?

    Thanks

    The answer is: yes.
    I resubmitted the same build which failed the certification (didn't upload anything new) and now passed it.

    Marco

    Tuesday, October 16, 2012 11:33 PM

All replies

  • For the record, my app passed the "Windows App Certification Kit" of course

    Wednesday, October 10, 2012 9:54 AM
  • It seems you've been provided specific information to act upon, which I think is a huge improvement over the past - previously cert failure feedback just pointed you to the requirements concerned.

    Issue #1 : indeed you must provide test credentials for the store testers to exercise your app's full functionality. Testers won't use their personal FB account for testing so my take is you do need to create a test account for them to use - which is quite easy to do in FB : log into your FB account, go to https://developers.facebook.com/apps, click 'Edit Roles' and then click 'Create' in the Test Users section. You may need to configure permissions depending on what your app is doing.

    Issue #2 : in addition to app launch and suspend, this requirement checks your app's memory working set. You should run your app in the profiler and check how much memory your app consumes across your different usage scenarios.  Or you can just start with the task manager and look at your app's memory usage as you work the app. If you can identify which scenarios make the memory shoot up say past the 500MB bar, then you can look into how you can make these code paths more efficient.


    • Edited by ykhammal Wednesday, October 10, 2012 1:22 PM
    Wednesday, October 10, 2012 1:21 PM
  • #1 The Test Users accounts in fb are accessible only through the developer's owner... you cannot provide those credentials.
    About this, the only solution that it seems possible is creating a REAL FAKE account on facebook. I don't know how much Facebook will be happy about this when it discover that Microsoft Developers needs to create fakes account to make their application validated.

    #2 With task manager, measuring the app in all conditions, the memory footprints never exceeded 35Mb.

    I don't know how to proceed...

    Thanks,
    Marco 

    Wednesday, October 10, 2012 1:52 PM
  • #1 I just created a facebook test user, changed his name and password, then clicked 'Switch To' link for that user, looked up the user's fake email address in his account, then logged out and logged back in with the test user's email and password, it worked. That should work for testers too, though you probably need to ensure the test account remains valid for long enough.

    #2

    - did you go through all possible usage scenarios ?
    - Did you test supend and resume in Visual Studio ?
    - Do you save and restore app or page state anywhere ? Did you modify startup code ?

    Otherwise you should probably use the profiler to investigate further


    • Edited by ykhammal Wednesday, October 10, 2012 2:11 PM
    Wednesday, October 10, 2012 2:10 PM
  • Thanks for the assistance through the problem #1, I figured out how to create and use a testing account ;)

    #2 Unfortunately yes, I tried everything..
    Moreover, my app has only 3 page... so it's not complicated to try every possible scenario.
    I'll try with the profiler and I let you know... thanks!

    Marco

    Wednesday, October 10, 2012 2:54 PM
  • Tried everything (every page, suspending, snapping, background task, clicking everywhere on insane mode), with the performance analysis open.
    Unfortunately it measures only the CPU but I kept an eye to the allocated ram with the task manager (which never went over 53Mb).

    I used a debug version of the app, so the release one should be lighter.

    I didn't notice anything wrong, cpu always very low.
    Here the performance report: FbFriends121010.vsps .

    Any suggestion?

    Thanks,
    Marco

    • Edited by Marco Mura Wednesday, October 10, 2012 3:49 PM url fixed
    Wednesday, October 10, 2012 3:34 PM
  • Hard to read your perf report since I don't have the symbols, but it looks like you're using the store API. Have you removed all references to CurrentAppSimulator from your code ?
    Wednesday, October 10, 2012 3:49 PM
  • Yes of course... the only methods where I call CurrentApp[Simulator] are these:

            public static bool IsTrial()
            {
    #if DEBUG
                return CurrentAppSimulator.LicenseInformation.IsTrial;
    #else
                return CurrentApp.LicenseInformation.IsTrial;
    #endif
            }
    
            public static async Task RequestAppPurchaseAsync()
            {
    #if DEBUG
                await CurrentAppSimulator.RequestAppPurchaseAsync(false);
    #else
                await CurrentApp.RequestAppPurchaseAsync(false);
    #endif
            }
    

    You may see CurrentAppSimulator class because I ran the analysis in debug mode, but in release I use only the CurrentApp one.
    Wednesday, October 10, 2012 3:52 PM
  • Agree, 9 hours ago my app fail too, same reason, same message. And more, many people have failed tests 9 hours ago, with some reason, for example http://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/f3b92014-e50e-4b20-861c-ff0c2c8b36dc.

    It looks like there was some bug in test system, or maybe some tester make an error, and he fail all apps.

    I am 100% sure my app have good launch and suspend time, becouse my app is already in store, it was 5 version, i doesnt change anything in launch/suspend.

    Wednesday, October 10, 2012 4:10 PM
  • Agree, 9 hours ago my app fail too, same reason, same message. And more, many people have failed tests 9 hours ago, with some reason, for example http://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/f3b92014-e50e-4b20-861c-ff0c2c8b36dc.

    It looks like there was some bug in test system, or maybe some tester make an error, and he fail all apps.

    I am 100% sure my app have good launch and suspend time, becouse my app is already in store, it was 5 version, i doesnt change anything in launch/suspend.

    I'm in the same position.  My app failed after six days of being in the "content compliance" phase for supposedly failing the 3.8 suspend/resume guideline.  Isn't it odd that the app would fail for an automated test after being in the "content compliance" phase for six days?
    Wednesday, October 10, 2012 4:12 PM
  • So? What should we do?
    Resubmit the app, waiting for the certification team to fix server-side the submission?

    Thanks

    Wednesday, October 10, 2012 6:31 PM
  • So? What should we do?
    Resubmit the app, waiting for the certification team to fix server-side the submission?

    Thanks

    The answer is: yes.
    I resubmitted the same build which failed the certification (didn't upload anything new) and now passed it.

    Marco

    Tuesday, October 16, 2012 11:33 PM
  • So that means there was indeed a problem in the store ? It would be better if Microsoft were upfront about it and notify people when that happens
    Thursday, October 18, 2012 9:57 AM