Pex Forum

Shouldn't Pex-generated test fail?

Tue, 24 Nov 2009 10:21:39 Z

public static IEnumerable<A> GetAs(IEnumerable<Base> bases)

{

    Contract.Requires(bases != null);

    Contract.Requires(Contract.ForAll(bases, x => x != null));

    Contract.Ensures(Contract.Result<IEnumerable<A>>() != null);

    Contract.Ensures(Contract.ForAll(Contract.Result<IEnumerable<A>>(), a => a != null));



    return from item in bases let a = item as A where a != null select a;

}



public static IEnumerable<A> GetAs()

{

    Contract.Ensures(Contract.Result<IEnumerable<A>>() != null);

    Contract.Ensures(Contract.ForAll(Contract.Result<IEnumerable<A>>(), a => a != null));



    var list = new List<Base> { null, null };

    return GetAs(list);

}

The Pex (0.19.41110.1) generates tests and means that the second method would work! Obviously code contracts of the first method are violated when the first method is called from the second method. Following classes are used:

abstract class Base {}

class A : Base {}

class B : Base {}

Moles and Stubs vs. Moq

Sun, 15 Nov 2009 23:41:05 Z

Given that Moles and Stubs can accomplish a lot, doesn't it make sense to add the remaining Moq features as well to Pex? if not, why?

How to refactor tests that use AsStub helper

Tue, 24 Nov 2009 23:37:51 Z

From the "0.19" release notes:

No more AsStub and AsMole: the AsStub and AsMole helpers were confusing and pretty useless so we decided to reduce the amount of generated code by removing them.

We have some tests that use the AsStub helper. How do you advise updating these tests to work with the 0.19 release?

Thank you!

Creating Parameterized Moles returns null Mole only

Mon, 23 Nov 2009 15:29:17 Z

I was working on a demo on using Pex with SharePoint and stumbled upon the following challenge.

Imagine the following code:

public static SPListItem GetNewsMessage(string title)
{
    SPListItem newsMessage = null;

    if (String.IsNullOrEmpty(title))
    {
        throw new ArgumentException("title");
    }

    SPList newsList = SPContext.Current.Web.Lists["Pages"];
    if (newsList != null)
    {
        SPListItemCollection newsItems = newsList.GetItems(new SPQuery()
        {
            Query = String.Format("<Where><And><Eq><FieldRef Name='ContentType'/><Value Type='Text'>News</Value></Eq><Eq><FieldRef Name='Title'/><Value Type='Text'>{0}</Value></Eq></And></Where><OrderBy><FieldRef Name='Created' Ascending='True'/></OrderBy>", title),
            RowLimit = 1
        });

        if (newsItems != null && newsItems.Count > 0)
        {
            newsMessage = newsItems[0];
        }
    }

    return newsMessage;
}

It's a method that returns a list item with the given title.

The problem is that Pex doesn't generate any test to cover the case when an item is found (ie. newsItems != null && newsItems.Count > 0). The mock of the GetItems(SPQuery) method always results in a null value. While I could create two PexMethods to cover this I'd rather use a single method with parameterized mole.

I've looked at the documentation provided with Pex and came up with the following PexMethod:

[PexMethod]
public SPListItem GetNewsMessage(string title)
{
    MSPContext.FallbackAsNotImplemented();

    MSPContext.CurrentGet = () => new MSPContext()
    {
        WebGet = () => new MSPWeb()
        {
            ListsGet = () => new MSPListCollection()
            {
                ItemGetString = listName =>
                {
                    PexAssert.IsTrue(listName == "Pages");
                    return new MSPList()
                    {
                        GetItemsSPQuery = query =>
                        {
                            PexAssert.IsNotNull(query);
                            var call = PexChoose.FromCall(this);
                            return call.ChooseResult<MSPListItemCollection>();
                        }
                    };
                }
            }
        }
    };

    SPListItem result = RecentNewsWebPart.GetNewsMessage(title);
    return result;
    // TODO: add assertions to method RecentNewsWebPartTest.GetNewsMessage(String)
}

Unfortunately, as I mentioned before, the GetItemsSPQuery method always returns a null value.

Am I missing something here, or is this something SharePoint-specific that isn't supported by Pex yet?

w: http://blog.mastykarz.nl | t: @waldekm | c: http://imtech.codeplex.com | c: http://www.imtechvelocity.nl

Pex doesn't appear to honor PexAllowedException attribute.

Tue, 10 Nov 2009 15:32:59 Z

I have some test that were throwing exceptions that were expected, and I used the Allow Exception option in the Pex Exploration Results window, and I can see my code that it now reads:
[PexMethod, PexAllowedException(typeof(NullReferenceException))]

public UpgradeType CheckRequiredParameters(CommandLineArguments parsedArguments)

{
...

but when I run the test again using MSTest I still get NullReferenceExceptions. Am I missing something to make the exception allowed? I'm using Pex 0.18.41014.0, and VS 2008 Team Suite.

Thanks,
--Geoff

Thanks, --Geoff

How to mock a SerialPort with events?

Tue, 10 Nov 2009 19:37:19 Z

Hi, me again.

I'm still trying pex on the application involving communications through SerialPorts.

Now I'm faced of the case that when SerialPort.write is used to send a command, we know what is the expected return message if the other side of the communication is working properly. And my program simply listen to the DataReceived event of the port and act corresponding.

So that I wrapped the write() method of SerialPort and want to raise a DataReceived event at the end to notify the program under test, it looks like:

//WriteString is the mole of SerialPort.write(String s)
MSerialPort.AllInstances.WriteString = (writer, s) => {
    //do something that I need to map the command s with its expected response;
    //TODO: raise a DataReceived event so that the handler of the program can be invoked.
};

Currently I'm applying a wrapper of the handler in my test (because it is private) so that I can invoke it in my PUT, but that does not seem a good solution since it provides a potential to change the behavior of the program under test.

Would you sharing some ideas about this problem?

Thanks.

Does mole generation skip abstract properties and methods of an abstract class?

Sat, 21 Nov 2009 22:39:29 Z

Does mole generation skip abstract properties and methods of an abstract class? How to enable that? I cannot use stubs for this task because i need to change

sealed methods as well.

Error while executing a test from the Test View: Host process failed with exit code ClrMonitorFail

Mon, 23 Nov 2009 09:40:55 Z

Executing a test from the Test View window results in the following exception:

Error 11/23/2009 10:35:31 AM !error! [agent] host process failed with exit code ClrMonitorFail (-667) WINDOWS7
Error 11/23/2009 10:35:31 AM !error! [agent] host exited before connecting WINDOWS7
Error 11/23/2009 10:35:31 AM !error! [agent] failed to start Pex Host for GetNewsMessage_NewsFound_SPListItem03 (in ....\projects\pextest.classlibrary7\classlibrary7.tests\bin\debug\classlibrary7.tests.dll) WINDOWS7

This exception occurs only when the Code Coverage has been enabled in Local.testsettings.

w: http://blog.mastykarz.nl | t: @waldekm | c: http://imtech.codeplex.com | c: http://www.imtechvelocity.nl

Adding Precondition doesn't take the framework version into account

Mon, 23 Nov 2009 10:31:59 Z

Adding Precondition to a 3.5 Framework Assembly, adds the Contract.Requires statement which belongs to the 4.0 Framework.

w: http://blog.mastykarz.nl | t: @waldekm | c: http://imtech.codeplex.com | c: http://www.imtechvelocity.nl

Meaning of "Allow Exception..." menu item

Thu, 19 Nov 2009 04:54:43 Z

I got the following feedback from Manuel Fahndrich. In addition to adding a PexAllowedExceptionAttribute, it should also re-classify all generated test cases, turning "unexpectedly raised exceptions" into "expected exception".

Does anyone else have opinions about this feature in Pex?

Nikolai Tillmann - Tell us how you use Pex

Pex always crashes with StackOverflow

Sat, 21 Nov 2009 14:04:18 Z

Hi,

after running Pex on a method it constantly crashes. A window pops up saying "Microsoft Pex Exploration Host funktioniert nicht mehr" (in english something like: "Microsoft Pex Exploration Host doesn't work anymore"). The error message in the Pex Exporation Results Window says: "Review Bold Issuses: (!) StackOverflow". This happens every time I run Pex on a method, even in very simple test applications (empty console application with just a single method).

I don't know if there is a correlation but it seems to me, as if the problem appeared after installing Code Contracts. But now even after uninstalling Code Contracts the problem persits.

I am using: Windows 7 (32bit), VS 2008 Proffesional, Pex 0.19.411101.1

Thank You
Michael

Stubs not working after Upgrade from 0.17 to 0.19

Fri, 20 Nov 2009 15:47:13 Z

I have delete the old stub file and added a new one:

<?xml version="1.0" encoding="utf-8" ?>
<!--
This file is used by Microsoft.Stubs to generate stubs and moles
for all the types in the assembly under test.
-->
<Stubs xmlns="http://schemas.microsoft.com/stubs/2008/">
  <Assembly Name="Tracker" />
</Stubs>

When i compile or do "Run custom Tool", the generated .designer.cs is empty. No error messages related to stubs are shown. However the project containing the .stubx currently does not compile. What am i doing wrong here?

Stubs and Moles generation for internal visible classes

Fri, 20 Nov 2009 13:01:41 Z

Hello,

With the latest version of Pex (v0.19.41110.1 DevLabs Pre Release for commercial evaluation) some stuff started working differently. Particularly with the latest version I cannot get stubs and moles for internal classes visible for a test project.

The case:
- a class library "ClassLibrary" has two classes: one public and one internal;
- "ClassLibrary" has an attribute applied in the AssemblyInfo.cs
    [assembly: InternalsVisibleTo("TestProject")]
- "TestProject" is my VS test project;
- "TestProject" has ClassLibrary.stubx with the following content
    <Stubs xmlns="http://schemas.microsoft.com/stubs/2008/">
      <Assembly Name="ClassLibrary" />
    </Stubs>
- Stubs generate stubs and moles only for the public class from "ClassLibrary".

Stubs v0.18.41014.0 definitely generated stubs/moles for my internal classes visible inside a test project.

Is it a bug or stubs/moles generation of internal visible classes was removed?

Thanks.

--Yauhen

New to Unit Testing

Thu, 19 Nov 2009 22:48:46 Z

Hi, I have a couple of questions.

Say I have a person class:

Person
{
private string firstName;
private string lastName;
public string GetName()
{
return lastName + ", " + firstName;
}

I really want to run this against the "Person" object, so I want PEX to generate random people with random names and check the output. Obviously this is a simple example.

PEX seems to just create a new Person {} object, i.e. everything set to null. I kind of expected this, however, I would like to specify some Person objects of my own. Can I add additional cases over the ones that PEX finds?

Let me know if I'm doing it wrong

Regards,

sjh

Disable Partial Generation

Thu, 19 Nov 2009 15:01:44 Z

Is it possible to have PEX generate non-partial classes? I'd like to generate using PEX once and then hand craft my tests.

pex host type and vs2010

Tue, 17 Nov 2009 10:25:47 Z

hello
ive been having some trouble getting my moles to work again after moving to 2010, i found in the docs that only vs2008 is supported but is this still true for 0.19?
since the mole api has changed a little im not sure if im doing some misstake and thats why my moles doesnt get called, or if its because vs2010 is not playing nice with the pex host

VS2010 Issue

Wed, 18 Nov 2009 16:06:24 Z

I must have done something incorrect. I have a Win7 dev machine (x86) with both Visual Studio 2008 professional and Visual Studio 2010 Ultimate Beta 2 installed. I installed the latest version of Pex this morning and am able to successfully use it with VS2008. With VS2010, anytime I use Pex (with either an exisiting or new project) I get this message in the output:

launching pex

starting...

C:\Users\sandrews\documents\visual studio 2010\Projects\TestApplication\TestApplication\bin\Debug\TestApplication.exe

"C:\Program Files\Microsoft Pex\bin\Microsoft.Pex.exe" "C:\Users\sandrews\documents\visual studio 2010\Projects\TestApplication\TestApplication\bin\Debug\TestApplication.exe" /membernamefilter:M:Capitalize! /namespacefilter:TestApplication! /typefilter:Helper! /explorationreflectionmode:Wizard /x64failsilently /assemblyresolutiondirectories:"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\PublicAssemblies" /targetclrversion:v2.0.50727 /targetclrversion2 /donotopenreport /reportlevel:None /reportrootpath:"C:\Users\sandrews\documents\visual studio 2010\Projects\TestApplication\TestApplication\bin\Debug\reports" /testassemblyname:TestApplication.Tests /testframework:VisualStudioUnitTest /testlanguage:cs /testprojectfile:"C:\Users\sandrews\documents\visual studio 2010\Projects\TestApplication\TestApplication\TestApplication.csproj" /testprojectnotupdate /testprojectskip

launching new pex process

starting monitored process

listening to monitored process (cold start)

launched Pex 0.19.41110.1 x86 Edition on .NET v2.0.50727

00:00:00.0> starting execution

00:00:00.0> reflecting tests

!error! [metadata] bad image format encountered while loading explorations

[symbols] could not load symbols for C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorlib.dll

[symbols] search path:

00:00:00.8> [finished] execution time 00:00:00.7620000.

-- 0 critical errors, 1 errors, 0 warnings.

-- 0 generated tests, 0 failing, 0 new, 0 inconclusive.

[coverage] skipping coverage reports...

[reports] skipping html reports

EXPLORATION FAILED

monitored process exited with ReflectionError (-1011)

finished

Question Regarding PexSymbolicValue.GetRelevantInputNames

Tue, 10 Nov 2009 05:00:56 Z

Hi,

I was trying to use the method PexSymbolicValue.GetRelevantInputNames to get the inputs that influence the value of a variable.

It seems that the method returns the inputs considering both control-flow and data-flow.

Is it possible to get inputs based on control-flow only and data-flow only?

In addition, if an array A is an input and value of a variable V is dependent only on the first (say) element of the array A,

the method PexSymbolicValue.GetRelevantInputNames(V) returns "A" and "$Items".

Is it possible to precisely get the element of A (i.e., A[0]) on which V depends?

Best Regards,

Kunal

Blogging about Pex, let us know!

Wed, 20 May 2009 14:22:47 Z

If you are (or you've been) blogging about Pex, let us know and drop the link in this thread. We are building a list of links that will be hosted our project web site.

Your feedback is crucial for us!

Pex: Visual Studio Addin 0.19.41110.1 crashed with System.InvalidOperationException

Fri, 13 Nov 2009 06:40:52 Z

Pex Visual Studio add-in crashes while starting Visual Studio.

Pex 0.19.41110.1 Release Visual Studio 10.0.21006.1 Bug Report

------------------------------------------

- Open Visual Studio 2010

- Open the Pex Exploration Tool Window

- Dock it to the bottom

- Make sure it's activated

- Close Visual Studio 2010

- Open Visual Studio 2010

- The exception occurs

------------------------------------------

exception details:

System.InvalidOperationException: The DocumentSite on a WindowFrame may only be set once

Microsoft.VisualStudio.Platform.WindowManagement.WindowFrame.set_DocumentSite(DocumentObjectSite

value)

Microsoft.VisualStudio.Platform.WindowManagement.WindowManagerService.CreateContentPane(String

pszName, Int32 fDockViewOn, Int32 fDocument, String lpstrMkDoc, UInt32 eCreateWindowFlags, UInt32 dwToolWinId, Object punkView, Object punkData, IServiceProvider pServiceProvider, IVsUIHierarchy pUIHierarchy, UInt32 vsid, Int32& pfDefaultPosition, Guid& rguidCmdUI, IVsWindowFrame& ppWindowFrame)

System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32

errorCode, IntPtr errorInfo)

at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(Int32

errorCode)

at Microsoft.VisualStudio.NativeMethods.ThrowOnFailure(Int32 hr, Int32[] expectedHRFailure)

at Microsoft.VisualStudio.Shell.Package.CreateToolWindow(Type

toolWindowType, Int32 id, ProvideToolWindowAttribute tool)

at Microsoft.VisualStudio.Shell.Package.CreateToolWindow(Type

toolWindowType, Int32 id)

at Microsoft.VisualStudio.Shell.Package.FindToolWindow(Type

toolWindowType, Int32 id, Boolean create, ProvideToolWindowAttribute

tool)

at Microsoft.Pex.VsPackage.PEX.InitializeExplorationExplorerWindow()

at Microsoft.Pex.VsPackage.PEX.IntializePackage()

w: http://blog.mastykarz.nl | t: @waldekm | c: http://imtech.codeplex.com | c: http://www.imtechvelocity.nl

how to generate MDateTime class?

Mon, 16 Nov 2009 03:36:54 Z

how to run the code? Can't find MDateTime, if i must change the PexAssemblyInfo.cs file, and so, how to do?

Pex generates empty stub file for one project

Mon, 09 Nov 2009 16:38:02 Z

hello guys
all of a sudden pex doesnt generate any stubs for one of my projects. i havent changed the stubx file and it works fine for other projects in the same solution but not this one. i havent changed anything in the project when it stopped working except that i added a reference to another project, but even after i removed that reference i only get the following in the generated file:

// <auto-generated/>
#pragma warning disable 0067, 0108, 0618
[assembly: global::Microsoft.Stubs.Framework.StubsAssembly("CoreNetLib")]

the stubx file looks like this

<?xml version="1.0" encoding="utf-8" ?>
<!--
This file is used by Microsoft.Stubs to generate stubs for all the interfaces in the assembly under test.
-->
<Stubs xmlns="http://schemas.microsoft.com/stubs/2008/  <Assembly Name="CoreNetLib" />
</Stubs>

">

i dont get any errors from the stub generation either, acording to the stub output its doing the generation, it just results in nothing. all projects in the solution build.

is there any other place i can look for errors?

NullReferenceException - bug in Pex?

Thu, 12 Nov 2009 23:29:10 Z

I have a PUT that, when run under Pex Exploration, throws a NullReferenceException:

As you can see, the stack trace is rather unhelpful, it doesn't show where the actual error is occurring. Clicking "Go To" just takes me to the PUT, and that PUT isn't throwing any null reference exceptions.

Here's my PUT:

[PexMethod]
public void CommitCurrentTransactions_CausesSharedItemsToBeFlushedToDisk(
    [PexAssumeUnderTest, PexAssumeNotNull] SharedIndexing target)
{
    var sharedIds = new[] { Guid.NewGuid() };
    target.SetSharedState(sharedIds, true);
}

And here's SetSharedState:

public void SetSharedState(IEnumerable<Guid> ids)
{
     // Simplified to bare minimum to repro the problem.
     var unshared = WhereNotShared(ids);
     foreach (var id in unshared)
     {
          Console.WriteLine(id);
     }
}

As you can see, it merely Calls into WhereNotShared and evalutes the result.

Here's WhereNotShared. This is where the exception occurs. Commenting out this function fixes the bug.

private IEnumerable<Guid> WhereNotShared(IEnumerable<Guid> ids)
{
    var commandText = string.Format("SELECT Id FROM {0} WHERE ID = @IdParam;", tableName);
    using (var readCommand = new SqlCeCommand(commandText, connection))
    {
        var idParameter = new SqlCeParameter() { ParameterName = "IdParam" };
        readCommand.Parameters.Add(idParameter);
        foreach (var id in ids)
        {
            idParameter.Value = id;
            if (readCommand.ExecuteScalar() == null)
            {
                yield return id;
            }
        }
    }
}

Commenting out the "if (readCommand.ExecuteScalar() ..." line above fixes the bug.

My question is, where is the bug here? Is this a Pex bug? Bug in SQL CE? Help!

Tech, life, family, faith: http://judahgabriel.blogspot.com

Pex is not able to generate required input values to meet a condition, when using Factory method, whereas the required inputs are generated when the factory method code is moved to PUT

Thu, 12 Nov 2009 04:05:46 Z

Hi,

I have the following code under test, in which I have a branching condition that when met implies Pex goal is reached

public void TestMe(Utility util)
{
string value = "";
if(util.ContainsKey("name"))
{
   Params.Add("name",util.GetParam("name")        
   value = Param["name"]
}
if (value.Contains("--"))
 {
   PexGoal.Reached("ReachMe");
   return null;
 }
}

I have a factory method for the "Utility" object as below

public static Utility Create([PexAssumeNotNull]string sortString, [PexAssumeNotNull]string name)
 {
     Dictionary<string, string> input_dictionary = new Dictionary<string, string>();
     input_dictionary.Add("name", name);
     Utility util = new Utility(input_dictionary);
     return util;
 }

On applying Pex on a PUT that invokes TestMe, Pex was not able to achieve the goal. However, when I moved the Factory method code to the PUT, Pex was able to achieve the goal.

Just curious why...

Thanks!

Madhuri

Question about reporting object creation issue

Thu, 12 Nov 2009 21:00:19 Z

Could you please tell me the current mechanism Pex use to report an object creation issue? In what situation it will consider failing to flip a path condition is because of an object creation issue?

TearDown disposables with Pex?

Fri, 13 Nov 2009 18:41:24 Z

My class under test is an IDisposable object.

public class Foo : IDisposable
{
   ...
}

public static partial class FooFactory
{
    [PexFactoryMethod(typeof(Foo))]
    public static Foo Create()
    {
           ....
    }
}

Foo also creates a file on hard disk.

After each foo is created, I need to call .Dispose on it, then delete the file it created on hard disk.

How can I instruct Pex to do this for every Foo it creates through the factory?

Tech, life, family, faith: http://judahgabriel.blogspot.com

Do I use NUnit Assert or PexAssert inside my PUTs?

Fri, 13 Nov 2009 15:38:14 Z

I'm using Pex with NUnit. In my PUTs, can I use NUnit.Assert? Or must I use PexAssert class?

Tech, life, family, faith: http://judahgabriel.blogspot.com

Capitlize Method, Code Coverage different !!

Sun, 11 Oct 2009 13:16:49 Z

Hello PEX Team,

I take the Capitalized method that you represent in your PEX video, and I run pex through it, now when I check the code coverage it tells me that code coverage is 83% !! where in your video code coverage was 100% !!
I took the same code, so why my code coverage is different than yours ?

thanks

Make object creation through Reflection when an Invariant is found optional

Fri, 13 Nov 2009 09:27:18 Z

The module that decides to use Reflection to set the field of an object when a class invariant is defined on the should be refactored as an attribute.

Jonathan "Peli" de Halleux - Give us your input about Pex!

Regarding Number of blocks in Pex Report

Thu, 12 Nov 2009 14:58:59 Z

Hi,

I have a method as follows:

public override int Count
{
get { return references.Count; }
}

I am running both VSTS and Pex on this method. While VSTS shows that this method has 3 blocks, Pex shows that this method has only 2 blocks. Can you please tell me why the number of blocks in Pex's report and VSTS coverage results do not match?

Thanks!

why not auto get interface subs?

Thu, 12 Nov 2009 07:25:15 Z

public interface IOrderDAL {
    void GetOrderByID( Order order );
}

public class OrderBLL {
    private IOrderDAL _dal;
    public OrderBLL( IOrderDAL dal ) {
            if ( null == dal ) {
                throw new ArgumentNullException( "dal" );
            }
            _dal = dal; 
    }
}

then test Constructor. pex generate the code:

[PexMethod]
        public OrderBLL Constructor( IOrderDAL dal ) {
            // TODO: add assertions to method       OrderBLLTest.Constructor(IOrderDAL)
            OrderBLL target = new OrderBLL( dal );
            return target;
        }

but the pex result only one test, the generate test code is:

[TestMethod]
[PexGeneratedBy(typeof(OrderBLLTest), IsCustomInput = true)]
[ExpectedException(typeof(ArgumentNullException))]
public void Constructor02()
{
    OrderBLL orderBLL;
    orderBLL = this.Constructor((IOrderDAL)null);
}

then i add code to PexAssemblyInfo.cs:

[assembly: PexExplorableFromConstructor( typeof( OrderBLL ), typeof( SIOrderDAL ) )]

but no change, the result is not change? why?

and how generate a not null IOrderDAL to OrderBLL Constructor? thanks!

How do you use Pex in Visual Studio? Your Input Is Needed!

Tue, 08 Sep 2009 23:05:04 Z

In continuation of our previous thread about licensing questions (http://social.msdn.microsoft.com/Forums/en-US/pex/thread/52763686-3eb6-47cf-950d-21b2baa07423), we would like to get some input from everyone how you are using Pex, and how you would like to use it.

Please copy&paste and fill out the questions below.

1. Are you using Pex already? A) From Visual Studio, B) the command-line, C) other, D) no
2. Which version of Visual Studio are you using? A) Professional, B) Team System, C) other
3. How many people are in your team?
4. Do you write A) Parameterized Unit Tests (PUTs), or B) only use the Wizard-generated PUT stubs?

Any other comments are appreciated as well, about the license terms, or required Visual Studio versions.

Thanks,
Your Pex team

Pex to test Entity Framework

Tue, 10 Nov 2009 17:19:04 Z

Hi

I have a data access layer made of a library with entity framework in, and a seperate library to group together business operations.

I want to test the the business operation layer, how can i use PEX/STUBS/MOLES for this. For example a very simple method i want to test is

public Publisher GetMM4Publisher() {
var item = (from p in _context.Publishers where p.Id == 4 select p).FirstOrDefault();
return item;
}

obviously i need to "mock" out the entity framework, its just that i am having difficultly understanding how to do this. I looked thought the examples included with PEX relating to EF but i could not see how it was relevant in this case. I assume I need to mole out

var a = new Stubs.MEntities();
a.PublishersGet = () => new ObjectQuery<Publishers>("",null);

I just dont know how to mole this out correctly, any help would be greatly appreciated, PEX has been great i just want to use if for all my testing scenarios :)

Thanks!!!!!!!

Stubs needs a better name

Thu, 05 Nov 2009 10:23:23 Z

Search for “Stubs” on stackoverflow and it is very hard to find any questions/answers about the “Stubs” framework rather then about Stubs/mocks in general.

Likewise on google.

Therefore it is hard to anyone that is learning “Stubs”, so can we have a better name please…

TestCleanUp executed after MaxWorkingSet boundary reached ?

Tue, 10 Nov 2009 21:39:59 Z

Hi,
I was wondering if the TestCleanUp is Executed if the MaxWorkingSet boundary is reached.
Since in one of my Test, I allocate a lot of data, and if the TestCleanUp isn't executed, then there is no way, that any of the next tests may be executed.
Anyway, in my (Pex)Testclass, the big Test gets executed, but the next one says 'exploration boundary max working set was reached with 2826 MB' (the boundary is set to 1500 on the class), so I was wondering if the TestCleanUp, which is supposed to cleanup is even executed, of if there is any chance that it wasn't/maybe was even stopped ...
Anyway, since I increased the boundary I will probably know with the next run.

Can't create properly Test project

Wed, 11 Nov 2009 15:09:20 Z

Hi,

This is my first project with Tests, and already I am having a problem. But I don't believe that it is big issue, but somehow I can't resolve it.

Anyway, I would like to use Pex and I started with GettingStarted video and created Capitalize method. The problem occurs when I create tests using "Save All" option in the Pex exploration results window. Test project is created and Pex methods also, but somehow they are not decorated with TestMethod and TestClass attributes. And also a reference to UnitTest framework is not added in the project automatically. I presume that this is not a big issue and that it only needs to be set somewherem but I just can't find the solution.

All my method look exactly like in the video tutorial. Except for this issue.

Here is my StringTest class:

namespace PexDemo
{
    [PexClass(typeof(Strings))]
    [PexAllowedExceptionFromTypeUnderTest(typeof(ArgumentException), AcceptExceptionSubtypes = true)]
    [PexAllowedExceptionFromTypeUnderTest(typeof(InvalidOperationException))]
    public partial class StringsTest
    {
        [PexMethod]
        public string Capitalize(string value)
        {
            // TODO: add assertions to method StringsTest.Capitalize(String)
            string result = Strings.Capitalize(value);
            return result;
        }
    }
}

And a part of ...g.cs class:

using System;
using Microsoft.Pex.Framework.Generated;
using Microsoft.Pex.Framework;

namespace PexDemo
{
    public partial class StringsTest
    {
[PexGeneratedBy(typeof(StringsTest))]
[PexRaisedException(typeof(NullReferenceException))]
public void Capitalize01()
{
    string s;
    s = this.Capitalize((string)null);
}
[PexGeneratedBy(typeof(StringsTest))]
public void Capitalize02()
{
    string s;
    s = this.Capitalize("");
    PexAssert.AreEqual<string>("", s);
}
[PexGeneratedBy(typeof(StringsTest))]
public void Capitalize03()
{
    string s;
    s = this.Capitalize("\0");
    PexAssert.AreEqual<string>("", s);
}
...
...
...
}

As you can see there are no TestMethod attributes.

Cheers :)

Pex Exploration throws ClrMonitorFail (-667)

Wed, 11 Nov 2009 15:44:47 Z

I encounter an odd failure while running Pex Exploration. It reports the following error:

Exit Code ClrMonitorFail
Exit Code ClrMonitor
Exit code returned when the ExtendedReflection profiler encountered a fatal error.
Value: -667

I see the issue in the following environment:
- Pex 0.18.41014.0 DevLabs Pre-Release for commercial evaluation;
- VS 2008 SP1 Team System;
- Windows Vista Enterprise.

The code is pretty basic.

Assembly ClassLibrary contains one empty class:

public class ClassLib
{
}

Another assembly CodeToTest contains the class:

public class ClassUnderTest {
    private static readonly ClassLib staticObject = new ClassLib();

    public int Sum(int x, int y) { return x + y; }
}

And the test assembly TestProject contains one test fixture:

[TestClass, PexClass(typeof(ClassUnderTest))]
public partial class ClassTest {
    [PexMethod]
    public void Test(int x, int y) {
        var t = new ClassUnderTest();
        Assert.AreEqual(x + y, t.Sum(x, y));
    }
}

When I run the Pex Exploration against the TestClass.Test method, I see the error mentioned above.

I also tried to run the exploration from the command line — the same result.

c:\test\PexTest\TestProject\bin\Debug>pex TestProject.dll
Microsoft Pex v0.18.41014.0 -- http://research.microsoft.com/pex
Copyright (c) Microsoft Corporation 2007-2009. All rights reserved.

instrumenting... launched Pex 0.18.41014.0 x86 Edition on .NET v2.0.50727
process id: 6628
current directory: c:\test\PexTest\TestProject\bin\Debug
[reports] recycling reports in background
00:00:00.0> starting execution
00:00:00.0> reflecting tests
00:00:00.4> TestProject
    00:00:00.4> ClassTest
      00:00:00.5> Test(Int32, Int32)
        LAUNCHER FAILED
error description: ClrMonitorFail (-667)

How can I get more information about the nature of the error?

I assume the failure may occur, because I have the system with quite restricted environment. Though, I tried to run VS with Administrator's privileges.

The issue dissapears, if I comment the static member:

//private static readonly ClassLib log = new ClassLib();

Any thoughts why it happens?

Thanks!

Uninstrumented Methods warning for Types/Assembly witch are instrumented

Wed, 11 Nov 2009 00:15:09 Z

Hi,
I noticed that there are some Types/Assemblies witch are listed a Uninstrumented, even if I add the PexInstrumentType/PexInstrumentAssembly flag for them.
For Excample
Type: Object, IntPrt, Type, GC, Array, StringBuilder, EventArgs, Exception
Assmebly: System, Microsoft.ExtendedReflection

Is there any reason for this ?
If yes, then why are they even listed as Uninstrumented, and we are givven the option to instrument them ?

PexBooleanAsZeroOrOne not honored when placed on test

Tue, 10 Nov 2009 15:35:47 Z

I had some issues with PexAssertFailedExceptions, and found a posting about using the PexBooleanAsZeroOrOne to get ride of the exception, but when I add this to my test case that was generated using Pex I still get the issue when I run the test using MSTest. I'm using the latest Pex bits, and VS2008 Team Suite. My test method looks like this:

[

PexMethod]

[

PexBooleanAsZeroOrOne]

public void AddRegisterTagstoConfigGetSet([PexAssumeUnderTest]CommandLineArguments target, bool value)

{

Thanks, --Geoff

PEX explorations fails

Fri, 06 Nov 2009 14:45:09 Z

I follow thorught the Capatalise tutorial, e.g.
create the class library,
added the code and compiled it,
Created paramaterised test stubs,
however Run Pex Explorartions results in am error "createion of fixutre of type AssemblyName.StringExtenrionsTest Fail"
--- Description
creation of fixture of type AnotherTry.StringExtensionsTest failed

creation of fixture of type AnotherTry.StringExtensionsTest failed

--- Exception details

System.Security.VerificationException: Operation could destabilize the runtime. at System.Void AnotherTry.StringExtensionsTest..ctor()

The Output window is as follows!!

launching pex

starting...

\\data\users\rdefeo\Visual Studio 2008\Projects\AnotherTry\AnotherTry.Tests\bin\Debug\AnotherTry.Tests.dll

"C:\Program Files\Microsoft Pex\bin\Microsoft.Pex.exe" "\\data\users\rdefeo\Visual Studio 2008\Projects\AnotherTry\AnotherTry.Tests\bin\Debug\AnotherTry.Tests.dll" /membernamefilter:M:Capitalize! /methodnamefilter:Capitalize! /namespacefilter:AnotherTry! /typefilter:StringExtensionsTest! /x64failsilently /targetclrversion:v2.0.50727 /targetclrversion2 /donotopenreport /reportlevel:None /reportrootpath:"\\data\users\rdefeo\Visual Studio 2008\Projects\AnotherTry\AnotherTry.Tests\bin\Debug\reports" /testframework:NUnit /testlanguage:cs /testprojectfile:"\\data\users\rdefeo\Visual Studio 2008\Projects\AnotherTry\AnotherTry.Tests\AnotherTry.Tests.csproj" /testprojectnotupdate /testprojectskip

using waiting pex process

waiting for monitored process

listening to monitored process (warm start)

00:00:00.0> starting execution

00:00:00.0> reflecting tests

00:00:00.3> AnotherTry.Tests

00:00:00.3> StringExtensionsTest

00:00:00.3> Capitalize(String)

!error! [execution] creation of fixture of type AnotherTry.StringExtensionsTest failed

!warning! [execution] could not generate any test in 0 runs

00:00:00.4> [finished] execution time 00:00:00.3906150.

-- 0 critical errors, 1 errors, 1 warnings.

-- 0 generated tests, 0 failing, 0 new, 0 inconclusive.

[coverage] skipping coverage reports...

[reports] skipping html reports

EXPLORATION FAILED

monitored process exited with DefaultFailure (-1000)

finished

I have already installed and removed and re-installed
Do you have any idea what the problem is??????