locked
how to configure multiple test dll in TFS build definition. RRS feed

  • Question

  • Hi I have have to configure multiple test dall as part of the TFS build server.

    I have to configure two test dll, "SDK.Core.TestAutomation.dll" and "Microsoft.Research.Wwt.Sdk.Core.Test.dll"

    I have single testsettings file for both dlls. These two assemblies uses same single test settings file, So i have configured that test settings file in build definition and cannot run using msdi.

    Test case works fine with build when i specify single test dall in Test Assembly file specification.

    But test cases are being ignored when i specify multiple test dall. I tried specifying the search pattern in different ways,

    1. SDK.Core.TestAutomation.dll&Microsoft.Research.Wwt.Sdk.Core.Test.dll

    2. SDK.Core.TestAutomation.dll&&Microsoft.Research.Wwt.Sdk.Core.Test.dll

    3. SDK.Core.TestAutomation.dll|Microsoft.Research.Wwt.Sdk.Core.Test.dll

    4.. SDK.Core.TestAutomation.dll,Microsoft.Research.Wwt.Sdk.Core.Test.dll

    I tried the above combination with duble quotes as well and none of the above worked. Test cases are not running as part of the build.

    Waht is the search pattern should i use here. I cannot use **\*test*.dll as it runs all the dll which has test in it.. I have some other dlls which i dont want to run , so i cannot use **\*test*.dll.

    And also, Can i use search pattern in test assembly file specification to configure multiple test dll which are using different test settings and are from different solution.

     

    Thanks,

    Shankar.

    Monday, February 21, 2011 9:48 AM

Answers

  • Hi HShankar,
    If you can't get a search pattern to work, you might look at other options:

    Multiple File Spec's
    You can look at adding a second Test specification.  In the Build definition's Process tab, you can add and remove test specifications from the "..." button in the Automated Tests field.  You might experiment with adding the tests individually per dll since there are only two. 

    Rename the Libraries
    If the default pattern (**\*Test.dll) is not sufficient, perhaps you can rename the libraries to establish a pattern that will work.

    Categories
    Categories can be used to limit the unit tests run.  If there is a logical category you can apply to tests that need to be run as part of the build, perhaps you can keep the *Test*.dll search pattern and just specify a category.  Unfortunately you will only be able to specify the Microsoft.VisualStudio.TestTools.UnitTesting.TestCategoryAttribute at the Test (method) level and not on the assembly or class.

    Build Template Customizations
    The Build Template looks pretty flexible for updating.  There are two main activities that run to execute the MSTest.  There is a FindMatchingFiles activity which is responsible for finding files that match your search pattern.  Next files are piped into the MSTest task.  You can explore customization options.  Two come to mind immediately, (1) you can replace the FindMatchingFiles activity with a custom activity that can parse more complicated regular expressions.  (2) You can send in a diliminated list of assembly names and replace the FindMatchingFiles activity with a simple Assign activity.  You can assign a string.split on your diliminated file list to the testAssemblies variable.  Evaluate what works best for you.  (2) sounds simpler but I'd hate to totally remove the FindMatchingFiles activity from your template.


    Good Luck,
    -Mike

     

     

     

    Tuesday, February 22, 2011 5:54 AM

All replies

  • Hi HShankar,
    If you can't get a search pattern to work, you might look at other options:

    Multiple File Spec's
    You can look at adding a second Test specification.  In the Build definition's Process tab, you can add and remove test specifications from the "..." button in the Automated Tests field.  You might experiment with adding the tests individually per dll since there are only two. 

    Rename the Libraries
    If the default pattern (**\*Test.dll) is not sufficient, perhaps you can rename the libraries to establish a pattern that will work.

    Categories
    Categories can be used to limit the unit tests run.  If there is a logical category you can apply to tests that need to be run as part of the build, perhaps you can keep the *Test*.dll search pattern and just specify a category.  Unfortunately you will only be able to specify the Microsoft.VisualStudio.TestTools.UnitTesting.TestCategoryAttribute at the Test (method) level and not on the assembly or class.

    Build Template Customizations
    The Build Template looks pretty flexible for updating.  There are two main activities that run to execute the MSTest.  There is a FindMatchingFiles activity which is responsible for finding files that match your search pattern.  Next files are piped into the MSTest task.  You can explore customization options.  Two come to mind immediately, (1) you can replace the FindMatchingFiles activity with a custom activity that can parse more complicated regular expressions.  (2) You can send in a diliminated list of assembly names and replace the FindMatchingFiles activity with a simple Assign activity.  You can assign a string.split on your diliminated file list to the testAssemblies variable.  Evaluate what works best for you.  (2) sounds simpler but I'd hate to totally remove the FindMatchingFiles activity from your template.


    Good Luck,
    -Mike

     

     

     

    Tuesday, February 22, 2011 5:54 AM
  • Hi Shankar,

     

    Thanks for your post.

     

    For this issue, have you took Mike’s suggestion?


    John Qiao [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, February 23, 2011 3:44 AM
    Moderator