Performance issue with parameters of multiple actions

Question

Hi,

I'm trying to test a software bus using Spec Explorer. When I run this machine (presented below) either I get out-of-memory error or exploration takes a long time (full night) and then I terminated it. The good news is that it works well for on-the-fly testing though. But I do not know when to terminate the on-fly-testing either.

The actions I listed here (pub, sub, unsub, getNextMsg) are highly related to each other, therefore I will prefer to keep them in one machine. Moreover, there are data parameters for these actions. And I prefer some of these actions get replicated in each trace because that's how the real usage of the bus will be.

Any comments on how to handle exploration time and memory issues?

machine OneInstancePubSubUnsubReadMsgScenario() : Main where ForExploration = false
{
   let SoftwareBusAdapterImpl conn, string in_sub_subject, string in_sub_subject_2, string in_pub_subject, string in_pub_subject_2, string in_unsub_subject
      where
      {.
         Condition.In(in_sub_subject, "*", "test.>", "test.*.>");

         Condition.In(in_sub_subject_2, "*", "test.>", "test.*.>");

         Condition.In(in_pub_subject, "test", "test.test2", "test.FILL.test3", "NoSub.test");

         Condition.In(in_pub_subject_2, "test", "test.test2");

         Condition.In(in_unsub_subject, "*", "test.>", "test.*.>");

      .}
   in

    new conn.SoftwareBusAdapterImpl;conn.Create;conn.Connect;
    (conn.Subscribe(in_sub_subject, _) ||| conn.Subscribe(in_sub_subject_2, _) ||| conn.UnSubscribe(in_unsub_subject, _) |||
     conn.Publish(in_pub_subject, _) ||| conn.Publish(in_pub_subject_2, _) ||| conn.GetNextMsg{2}); conn.GetNextMsg;conn.Disconnect;conn.Destroy
}

machine OneInstancePubSubUnSubReadMsgProgram() : Main where ForExploration = true
{
    (OneInstancePubSubUnSubReadMsgScenario;exit) || ModelProgram
}

machine OneInstancePubSubUnSubReadMsgTestSuite() : Main where TestEnabled = true
{
    construct test cases where strategy = "shorttests" for OneInstancePubSubUnSubReadMsgProgram()
}

Answer 1

Hi Dharma11,

Your exploration practically became infinite.
Usually this big space is not of interest, so a cord scenario is used to control the exploration.
Spec Explorer is very fast even on very big models, if you control the exploration to the important state space.
So there is no need to reduce your model (Only if a lot of unimportant states are modeled).
I would recommend to stop the exploration, after having say 20 states.
Now you can inspect the partial exploration and find the unimportant states,
and you can slice these states away with an improved scenario (or limit parameter domain, or use coverage-strategy etc.). If all states are important on-the-fly testing is the way to go.

Hope you find something

Answer 2

Hi,

Thanks. I'm also getting out-of-memory error during on-the-fly testing, which I thought is intended for large models with many states. During on-the-fly testing, I noticed that before generating tests, Spec Explorer keeps exploring the machine for a pretty long time. Afterwards it starts generating some tests and then it reports out of memory error.

I did analyze my wrapper and models for any memory leaks or creating too many object, but I do not have anything like that.

One basic question for me is that does on-the-fly testing enumerates all possible combinations from the scenario mentioned in my machine before generating tests?

Thanks,

Dharma

Answer 3

Hi Dharma,

there are some limitations. E.g. the steps per state must be finite everywhere in the machine to run OTF on:

http://msdn.microsoft.com/en-gb/library/ff793088.aspx

Maybe you hit one of these limits.