Test Overkill

Question

Hi,
If you create a new Project, add an UserControl and give it 2 properties, Pex generates 14 (7 for each property) tests.
Pex also gets outofbounds (MaxRunsWithoutNewTests).
even if the properties are empty.
This is just overkill, since there is no way that the properties (without code or with just local vars) react differently just because you create the object differently.

Or am I missing something ?

Answer 1

Yes this is a limitation of Pex. The code coverage provided by the object factories is also taken into account when Pex decides to emit a test case or not. To emit a mininum of test cases, it should not take into account coverage collected while running object factories.

I will see if something is possible to make this more fined grained so I'll keep this post open to track this.

---

Jonathan "Peli" de Halleux

Answer 2

Some info on this:

I run Pex(VS Plugin) overnight.

In the morning VS2008 had crashed (OOM) and restarted.

So I checked how many tests Pex generated.

Sadly Pex only generated 579 tests, running Pex again, showed that ~500 of these tests are duplicates!

That's exactly the overkill I taked about

Answer 3

One thing interesting about these 'Duplicate' tests is, that some(I only checked a few) of them only do differ, in the factory, and some are really duplicates.
The real Duplicates are probably tests from runs before, but the ones witch differ in the factory parameters are created in only one Pex run!

Answer 4

Hi Alexander,

Is this behavior still hapenning?

---

Jonathan "Peli" de Halleux - Give us your input about Pex!

Answer 5

Hi Peli,
It doesn't seems like the exact same thing is happening, but there are still some Duplicates, sometimes even from the same run, but as far as I can tell, they do have different bodies, so it's no the exact same behavior