Test passes in Debug not in Run mode

Question

I have a unit test written for webservice call and web method test. It works fine if i run the test with my localhost in both Run and Debug mode, but while I try to run this test with some external agent it always fails though if i debug this test with the external agent, it passes.

Here is what I found but don't know the solution. It always fails while try to communicate with the webservices located on different server with external agent. Works fine with localhost though!

Thanks in advance!

Answer 1

On 11/2/2011 12:46 PM, Load_Tester wrote:

> I have a unit test written for webservice call and web method test. It

> works fine if i run the test with my localhost in both Run and Debug

> mode, but while I try to run this test with some external agent it

> always fails though if i debug this test with the external agent, it passes.

>

 

You are not doing a unit test. You are doing a functional or integration

test.

 

http://www.artima.com/weblogs/viewpost.jsp?thread=126923

 > Here is what I found but don't know the solution. It always fails while

> try to communicate with the webservices located on different server with

> external agent. Works fine with localhost though!

>

 

If the test passed on localhost, maybe that's all you need. One would

assume that if it passed locally (a functional test) ran locally against

IIS and the Web service, then it should run fine on a remote Web server.

The only way the Web service wouldn't work would be if there is some

kind of mis-configuration between client and service.

 

I have never run a test on a remote Web service. If it passed the

functional tests locally, then that's as far as I needed to take it.

 

Answer 2

On 11/2/2011 12:46 PM, Load_Tester wrote:

> I have a unit test written for webservice call and web method test. It

> works fine if i run the test with my localhost in both Run and Debug

> mode, but while I try to run this test with some external agent it

> always fails though if i debug this test with the external agent, it passes.

>

 You say unit test.

 

A test is not a unit test if:

 

1) It talks to the database

 

2) It communicates across the network

 

3) It touches the file system

 

4) It can't run at the same time as any of your other unit tests

 

5) You have to do special things to your environment (such as editing

config files) to run it.

 What you are talking about is integration or functional testing.

 

> Here is what I found but don't know the solution. It always fails while

> try to communicate with the webservices located on different server with

> external agent. Works fine with localhost though!

>

 

If the Web services test passed on localhost and in debug mode, maybe

that's all you need.

One would assume that if it passed locally (a functional test) ran

locally against IIS and the Web service,

then it should run fine on a remote Web server, it doesn't need to pass

additional tests.

 

The only way the Web service wouldn't work would be if there is some kind

of mis-configuration between client and the remote Web service.

 

I have never run a test on a remote Web service. If it passed the

functional tests locally,

then that's as far as I needed to take it.

 

 

Answer 3

Hello

It is not important if your tests are strict unit tests or not. It is important that they are triggered by mstest and that means that framework should take care of them. Sometime you just cant change manufacture code because you are not in developers member so you have to cover method as it is and without additional tool you just cant break dependencies and use mocks etc. But still, mstest manages that kind of tests.

I have thousands of tests and only one test fails all the time in normal run. In debug it works perfect. Me and other testers in group cannot solve problem so its like an holy grail in solution :) i had some experience with that kind issue because i had many problems with that kind of run. Sometimes test which is more complicated and more complex than normal unit test and dependencies requires some time to i.e establish connection with db, write file on hard drive etc. So try sleep thread for about 5000ms after each action and check if that solves problem. If it does, try to reduce sleep duration to minimum.

If it still doesnt work properly, try to run visual studio as an administrator.

Hope this would be helpful

Answer 4

OK, I found the problem.

I have had a XML setting file stored in TFS, When I was debuging it, it looks for the local refn but while I was running it, it was looking for refn on agent server and was failing. I changed the refn and it is working fine now.