none
Issue with MTM - Test Case run order not behaving. RRS feed

  • Question

  • We are having an issue with MTM. We re-order our test cases for a suite from the Testing Center \ PLAN tab, but when we go to the TEST tab the reordering is not realized. It still displays and runs the test cases in the order that they were created (by work item #).

    I am using VS 2010 Ultimate V10.0.40219.1 SP1Rel

    Help please.

    Friday, June 24, 2011 2:00 PM

Answers

  • Gentlemen,

    We were able to correct the problem by bringing the Server Visual Studio up to the same release as the Client.

    We had have a test server and a production server. Our administrators upgraded the production server and we all upgraded our clients to Visual Studio 2010 SP1. The administrators had not upgraded the test server so when we worked with the new client against the old server, we had the problem. Once they upgraded the test server, the problem went away.

    Conclusion: Verify that your client and server have compatable releases.

    Thanks

    -Steve


    Steven Becker
    Thursday, July 21, 2011 2:07 PM

All replies

  • We are having an issue with MTM. We re-order our test cases for a suite from the Testing Center \ PLAN tab, but when we go to the TEST tab the reordering is not realized. It still displays and runs the test cases in the order that they were created (by work item #).

    I am using VS 2010 Ultimate V10.0.40219.1 SP1Rel

    Help please.


    Really? No one else is having this problem?

    More information: I have a Suite and then a sub-Suite under it. It is the sub-suite that I am trying to re-order. One of the test cases in the sub-suite is an Automated Coded UI test. I believe the suites are STATIC, I did not create them with a query. I don't know if any of this information helps. I still need help.

    Thanks,


    Steven Becker
    Tuesday, June 28, 2011 7:14 PM
  • In the Plan Tab, you can reorder test cases by clicking on the Order button (or by right-clicking on a test suite and selecting Order Tests).

    In the Test tab, you can sort by the Order Column.

    The test tab shows test cases grouped by the test pass/ fail status - active, pass, fail, blocked, in progress etc.

    Within these groupings, test cases are ordered by the value provided in the Order column in Plan tab.

    If you want to rerun a bunch of test cases with pass/ fail status, first you should mark them all as active again. Now, within the "active" group in test tab, they will be sorted in the expected Order. Now, you can rerun the test cases.

    Tuesday, July 5, 2011 3:06 PM
    Moderator
  • Shankar, thanks for your interst in my problem but my system is not behaving the same as your.

    I am expecting what you described but the Test Plan Order #'s are always the same sequence as the ID #'s.

    You stated:

    "Within these groupings, test cases are ordered by the value provided in the Order column in Plan tab."

    It is not working for me. Here are the steps I am taking to produce the problem:

    1.)    Open MTM and under the “Testing Center” “Plan tab”, Create a Suite called “AutoSetup”.

    2.)    In that Suite create three test cases. Call them Test Case One, Test Case Two and Test Case 3.

    3.)    They will have consecutive ID #’s. In my case ID 27, 28 and 29.

    Order     ID           Name

    1             27           Test Case One

    2             28           Test Case Two

    3             29           Test Case 3

    4.)    Now from the Plan Tab, reverse the order of the test cases

    Order     ID           Name

    1             29           Test Case 3

    2             28           Test Case Two

    3             27           Test Case One

    This is the order that I want to run the test cases when I run the suite.

    5.)    Navigate to the Test Tab and note the Order #’s associated with the ID #’s are not the same as the Plan Tab.

    6.)    Sort the Order Column anyway you want but the Order #’s are in the same order as the ID #’s which is the same order that they were created.

    7.)    This is what I now see in the Test Tab :

    Order     ID           Name

    1             27           Test Case One

    2             28           Test Case Two

    3             29           Test Case 3

     

    8.)    When I try to run the suite. It always runs the active test cases in the order that they were created! I want to run them in the reverse order.

    Do you see anything that I might be doing wrong? Do you have any ideas? Do you know anybody at MS that might be interested?

    Thanks

    -Steve


    Steven Becker
    Monday, July 11, 2011 6:37 PM
  • Monday, July 11, 2011 9:29 PM
  • Jeff, thanks for the interest.

    I am not using query based suite or a requirements based suite.

    Your screen shots are what I would expect and I would expect the test cases to run in the order shown in the Test Tab.

    My system is not behaving that way. As I noted above:

    Navigate to the Test Tab and note the Order #’s associated with the ID #’s are not the same as the Plan Tab."

    So the question becomes what is wrong with my setup?

    I am running Visual Studio 2010 on a 64 bit machine running Windows 7 Ultimate SP1 with 4.00 GB memory:

    Microsoft Visual Studio 2010
    Version 10.0.40219.1 SP1Rel
    Microsoft .NET Framework
    Version 4.0.30319 SP1Rel

    Installed Version: Ultimate

    Microsoft Office Developer Tools   01019-532-2002102-70858
    Microsoft Office Developer Tools

    Microsoft Visual Basic 2010   01019-532-2002102-70858
    Microsoft Visual Basic 2010

    Microsoft Visual C# 2010   01019-532-2002102-70858
    Microsoft Visual C# 2010

    Microsoft Visual C++ 2010   01019-532-2002102-70858
    Microsoft Visual C++ 2010

    Microsoft Visual F# 2010   01019-532-2002102-70858
    Microsoft Visual F# 2010

    Microsoft Visual Studio 2010 Architecture and Modeling Tools   01019-532-2002102-70858
    Microsoft Visual Studio 2010 Architecture and Modeling Tools
       
    UML® and Unified Modeling Language™ are trademarks or registered trademarks of the Object Management Group, Inc. in the United States and other countries.

    Microsoft Visual Studio 2010 Code Analysis Spell Checker   01019-532-2002102-70858
    Microsoft Visual Studio 2010 Code Analysis Spell Checker

    Portions of International CorrectSpell™ spelling correction system © 1993 by Lernout & Hauspie Speech Products N.V. All rights reserved.

    The American Heritage® Dictionary of the English Language, Third Edition Copyright © 1992 Houghton Mifflin Company. Electronic version licensed from Lernout & Hauspie Speech Products N.V. All rights reserved.

    Microsoft Visual Studio 2010 Team Explorer   01019-532-2002102-70858
    Microsoft Visual Studio 2010 Team Explorer

    Microsoft Visual Web Developer 2010   01019-532-2002102-70858
    Microsoft Visual Web Developer 2010

    Coded UI Test Editor   1.0
    Enable editing of a UITest (*.uitest) file

    Crystal Reports Templates for Microsoft Visual Studio 2010  
    Crystal Reports Templates for Microsoft Visual Studio 2010

    Microsoft Team Foundation Server 2010 Power Tools   3.1.30831.0
    Power Tools that extend the Team Foundation Server integration with Visual Studio.

    Microsoft Visual Studio 2010 SharePoint Developer Tools   10.0.40219
    Microsoft Visual Studio 2010 SharePoint Developer Tools

    Microsoft Visual Studio 2010 Ultimate - ENU Service Pack 1 (KB983509)   KB983509
    This service pack is for Microsoft Visual Studio 2010 Ultimate - ENU.
    If you later install a more recent service pack, this service pack will be uninstalled automatically.
    For more information, visit http://support.microsoft.com/kb/983509.

    Microsoft Visual Studio 2010 Visualization and Modeling Feature Pack   2.0
    Extends the visualization and modeling tools in Visual Studio Ultimate:  generate dependency graphs for Web sites and native projects, generate code from UML models, build UML models from Visual C# code, import UML from other tools, create and follow bidirectional links between work items and UML models, and add custom commands and validation code to layer diagrams.

    Microsoft Visual Studio Process Editor   1.0
    Process Editor for Microsoft Visual Studio Team Foundation Server

    --------------------------------------------------------------

    My co-workers have the same problem.

    Could it be a TFS server setup issue?

    Any ideas?

    Thanks again

    -Steve


    Steven Becker
    Tuesday, July 12, 2011 1:54 PM
  • I believe I am seeing the same thing. Below is what I am observing.

    Plan Tab:

    Order ID  Title
    1       1    CodedUI_1
    2       3    CodedUI_3
    3       6    CodedUI_6
    4      21   CodedUI_21
    5      19   CodedUI_19

    Test Tab:

    Order ID  Title
    1       1   CodedUI_1
    2       3   CodedUI_3
    3       6   CodedUI_6
    4      19  CodedUI_19
    5      21  CodedUI_21

    These are test cases witin a create Test Suite (not a query-based suite). The results above are observed when simply toggling between the two and doing a refresh. At first glance, tt looks like the Test Tab sets the order to the ID # in the Test Case Manager (Organize Tab).

    Thoughts on what I may be doing wrong?

    Joel A


    Tuesday, July 12, 2011 9:19 PM
  • I have the same problem.

     

    Have MTM patch applied as well.

     

    This is using a requirements based suite with multiple tests per requirement. So If I order the tests for that 1 requirement, navigate to the test pane, the order is incorrect.


     

    Wednesday, July 13, 2011 10:19 AM
  • So was there any traction on this? I'm experiencing the same issue.
    Wednesday, July 20, 2011 9:11 PM
  • Gentlemen,

    We were able to correct the problem by bringing the Server Visual Studio up to the same release as the Client.

    We had have a test server and a production server. Our administrators upgraded the production server and we all upgraded our clients to Visual Studio 2010 SP1. The administrators had not upgraded the test server so when we worked with the new client against the old server, we had the problem. Once they upgraded the test server, the problem went away.

    Conclusion: Verify that your client and server have compatable releases.

    Thanks

    -Steve


    Steven Becker
    Thursday, July 21, 2011 2:07 PM
  • Hello,

    we brought server and client to the same release (2010 sp1). and we still have the issue with the order.

     

    Regards,

    D.A.

    Tuesday, September 6, 2011 10:02 AM
  • I think what he meant was that the Test Controller and Test Agent also need to be patched to SP1 (Test Controller is patched to TFS SP1 and Test Agent to VS SP1).

    Have you verified the version of the test controller and agent?

     


    Jeff Levinson | ALM MVP
    Author, "Software Testing with Visual Studio 2010"
    Saturday, September 10, 2011 3:25 PM
  • I'm new to Test Manager and am experiencing the same issue.  How can I find the version of the test controller and the test agent?

    Thanks!

    Tuesday, April 10, 2012 6:02 PM
  • @CindyTest: The version of Test agent can be found on the Configuration Tool. There will be a about button in the lower left corner. Mine was 10.0.3... I ran the VS2010SP1 patch and it changed to 10.0.4...

    Unfortunately it did not change anything for me, test controller, test agent, MTM are all at the same version.

    Monday, August 27, 2012 1:09 PM
  • I see the same issue. My MTM, Test Controller and Agent are the same version.

    Harini Shankar

    Wednesday, January 9, 2013 3:32 PM
  • I had the same issue and able to fix it by doing the following:

    1. Uninstall Visual Studio(Ultimate) - all the associate tools, agent, controller, patches that are related to the VS Studio. (Leave the SP1 update)

    2. Reboot the machine

    3. Install the following in order:

    1. VS Ultimate 2010
    2. VS Feature Pack2
    3. VS10-KB983578
    4. VS10-KB2387011
    5. VS10-KB2403277
    6. VS Lab
    7. VS Controller
    8. VS Agent

    4. Reboot the machine

    5. Go to Test Manager->Plan then Test tab to check->Now it run in the order that you setup in the Plan tab.

    Notes: You might see test cases not in order at first, click the State column, it will arrangeD in order you set in the Plan tab.

    This apply to both Manual and Automate test cases.

    Friday, February 1, 2013 5:20 PM
  • Hi,

    I am observing this issue with Microsoft Test Manager 2013. I have Visual Studion 12.0.31101.0 .

    Any one has experience with MTM 2013. I guess what path Sethea.B has mentioned might have been included with MTM 2013. 

    But still I am seeing that issue.

    Regards,

    Chirag

    Friday, February 27, 2015 8:59 PM
  • I have Visual studio 2013 premium installed and I'm seeing the same issue. I have my tests configured to run using deploy and test work flow. For some reason tests are not executing in the order that I specified in the suite in MTM. Any help is really appreciated.

    Thanks,

    Noble

    Monday, March 9, 2015 7:13 PM
  • following!
    Sunday, July 19, 2015 11:19 PM