How to visualize output of where {. Combination ... .}

Question

The configuration action ... where {. Combination ... .} logic seems to be difficult to understand and master.

Is there any easy way to see the output of the where {. Combination... .} logic will generate without running the tests?

It will be difficult to tell from the test runs if the Combination configuration worked as intended. The test runs are invalid and misleading if the Combination logic is not working correctly as intended.

What is the best way to verify that the Combination logic is working as intended ?

Would it be possible and useful if a pane could be made to pop up (or some other means) to display samples of the combinations that would be generated by a where clause before running the tests?

Answer 1

Hi DonBaechtel, did you try exploration? You can simply select a machine from Exploration Manager and click explore, then a state machine graph will show up and you can verify the Combination logic there.

Answer 2

Using the state machine graph may work in some very simple cases, but frequently in a real applicatiuon the combination will be mixed in and amoung the rest of the machine states and may can be very difficult to discern.

I was looking for something far easier to check, like a simple list of the Combinations generated for a single where clause.

Any hope for an easy and straightforward way to do this?

Answer 3

Hi Don,

Your are right with the observation that currently there is no directly supported way to observe parameter combinations outside of the graph viewer, and this should be considered for the feature backlog. However, here are some ideas how you can deal with this scenario right now.

Take a look at the Parameter Generation sample project in the release as a starting point.

The first you will see when you explore any of the machines there is that the graph viewer has been configured to display all combinations on one transition. This makes the representation of the graph much more concise if you generate many combinations which will all lead to the same model state. This is achieved by setting the property ViewCollapseLabels to true in the associated view.

If you need to further prune down the display of combinations in the graph, you can also set the property ShowParameters to false. Then only the name of the action will appear in the graph.

Now if you want to inspect the generated parameters offline, or dump them to a file because the purpose is to use Spec Explorer just for data generation, here is a 'hack' how this can be done. Again using the parameter generation sample project, you will see that the model program contains empty rules (as the purpose of this project is just to demonstarte combinations), for example:

namespace PG {
    static class Model
    {
        [Action]
        static void AddJob(string name, int time, Frequency frequency)
        {
        }
    }
}

Modify this as follows:

            [Action(DefaultParameterExpansionPoint=ParameterExpansionPoint.OnEntry)]
            static void AddJob(string name, int time, Frequency frequency)
            {
                Console.WriteLine("{0},{1},{2}", name, time, frequency);
            }

Writing to console output in model execution will be displayed in VS Output window, tab 'Debug'. When exploring machine Pairwise, I will find this in in the Output window:

@$^,-1,Once
@$^,3600,Weekly
t.cmd,3600,Once
t.exe,3600,Daily
@$^,60,Daily
t.exe,60,Once
t.cmd,60,Weekly
t.exe,-1,Weekly
t.cmd,-1,Daily

Its essential for this 'hack' to add the parameter 'DefaultParameterExpansionPoint=ParameterExpansionPoint.OnEntry' to the Action attribute, otherwise the execution of WriteLine would happen before the combinatorial algorithm takes over, leading to generate all combinations not just pairwise.

Instead of writing to console, you could also write to a file:

  System.IO.File.AppendAllText("\\temp\\myparams",
                    string.Format("{0},{1},{2}", name, time, frequency));

Hope any of these ideas are helpful.

Wolfgang

Answer 4

THANKS.