locked
Test coverage summary in case test fails or stopped by user. RRS feed

  • Question

  • After execution of the test I miss an overview of the test coverage. I can’t see the number of the states and /or transitions covered and those not covered in the part of my model so far tested.

     

    I miss a summary like the following:

    System hits X states: Y of Z expected states (%) are covered

    System hits K transitions: L of M expected transitions (%) are covered

    System hits A requirements: B of C expected requirements (%) are covered

     

    In case of offline testing (TCG), I expect this kind of summary, especially when test fails and stops before 100% transition coverage (chemise postman).

     

    In case of online testing (OTF), I expect this kind of report when execution of test is stopped by the used or stopped because of an error.

     

    In actual version of SpecExplorer 3.5 results is very brief. Is there any way to generate this kind of reports?


    Thanks in advance

    Best regards

     

    • Edited by RachidP Sunday, September 11, 2011 7:58 AM
    Sunday, September 11, 2011 7:54 AM

Answers

  • Hi RachidP,

    What you requested are useful statistics, but unfortunately they're not included by current version of Spec Explorer yet.

    What you can do is probably writing a test result analyzer, taking the test log and original exploration result as input, and then calculate coverage information. Spec Explorer provides a well defined object model for exploration result (.seexpl file), so importing it into your analyzer shouldn't be too hard. You can refer to the following article as a reference:

    http://blogs.msdn.com/b/specexplorer/archive/2009/11/24/using-the-exploration-result-object-model-basics.aspx

    Hope this helps. We'll file your feature request into our backlog and consider taking it in our next release.

    Thanks!

    Yiming


    Thanks, Yiming
    • Proposed as answer by Yiming Cao Wednesday, September 14, 2011 8:00 AM
    • Marked as answer by Xiang LiModerator Saturday, October 8, 2011 3:13 AM
    Wednesday, September 14, 2011 7:59 AM

All replies

  • Hi RachidP,

    What you requested are useful statistics, but unfortunately they're not included by current version of Spec Explorer yet.

    What you can do is probably writing a test result analyzer, taking the test log and original exploration result as input, and then calculate coverage information. Spec Explorer provides a well defined object model for exploration result (.seexpl file), so importing it into your analyzer shouldn't be too hard. You can refer to the following article as a reference:

    http://blogs.msdn.com/b/specexplorer/archive/2009/11/24/using-the-exploration-result-object-model-basics.aspx

    Hope this helps. We'll file your feature request into our backlog and consider taking it in our next release.

    Thanks!

    Yiming


    Thanks, Yiming
    • Proposed as answer by Yiming Cao Wednesday, September 14, 2011 8:00 AM
    • Marked as answer by Xiang LiModerator Saturday, October 8, 2011 3:13 AM
    Wednesday, September 14, 2011 7:59 AM
  •  

     

    Hi Yiming,

     

    Thanks for your answer.

     

    Awaiting implementation of this feature in next releases, I will try to use your suggestion wrt using the (.seexpl file) in combination with test log analyzer.

     

    I will keep you informed.

     

    Best regards

     

    Rachid



    • Edited by RachidP Wednesday, September 14, 2011 10:59 AM
    Wednesday, September 14, 2011 10:57 AM