locked
Newly Deployed Load Test Rig Is Stuck in "Pending" Status RRS feed

  • Question

  • I have tried to deploy a test rig for our Load Testing efforts and I am continually stuck in a "Pending" status in the "Test Results" window. 

    My rig is setup as follows:

        1 Dev Box (Physical Machine, Windows XP SP3, 32 bit all Windows Updates applied)
            VSTEST 2008 SP1
            Dual Core 2GHz 
            2 GB RAM
            60 GB HDD (> 16 GB free)
        1 Controller (Virtual Machine, Windows 2K3 Server SP2, 32 bit
            VSTEST 2008 SP1
            Single Proc
            1 GB RAM
            16 GB HDD (> 9 GB free)
        1 Agent (Virtual Machine, Windows XP SP2, 64bit all Windows Updates applied)
            VSTEST 2008 SP1
            2 Proc
            2 GB RAM
            16 GB HDD (> 7 GB free)
        LoadTest DB on 500 GB SQL Server 2005 Instance

    I don't see any reason why I would be having problems...  I am just trying to get a simple login/logout script running.  This load test works fine if I run it locally on the dev box.  I have verified all the ports are open and I can configure the rig throught the "Administer Test Controller" dialog.  I have modified the .testrunconfig file to use the Controller.  I have verified the accounts have the correct access.  I have read the following posts on the forum (none fix my problem):

    Why loadtest "Pending"?
    http://social.msdn.microsoft.com/Forums/en-US/vstswebtest/thread/3c97f796-f169-4ee0-840b-ba2122db0060
    Distributed load test stalling
    http://social.msdn.microsoft.com/Forums/en-US/vstswebtest/thread/def9e918-e6dd-4828-9be7-65a669fc6279
    Load tests on remote Agent with Vista installed not working!
    http://social.msdn.microsoft.com/Forums/en-US/vstswebtest/thread/25418b4a-8b6d-4846-83cb-0ad3417318ac

    Any thoughts or suggestions?  When I tried to get the "verbose logging" for the controller service, I couldn't get the service started when I had "CreateTraceListener" set to "yes" (Error 1053:  The service did not respond to the start or control request in a timely fashion)...

    Thanks in advance for any/all assistance you can provide!  Would like to get this up and going soon.  I have a new employee joining who will be using this rig daily.
    Thursday, October 16, 2008 8:45 PM

Answers

  • I think the problem may be that the controller service (running as VSTS_CONTROLLER) actually needs to call back over .NET remoting to the dev box in order to copy files that need to be deployed, and that user has no permission to login to your dev box.   I'm pretty sure that if you ran both the controller and agent services as your domain account everything would work.    I suspect you don't want to do that for security reasons, but it might be worth trying as an experiment, and if that works then give the VSTS_CONTROLLER an account on your dev box (you may need to do the same for the VSTS_Agent user on the controller). 

    Also, you may find Durga Gorti's blog helpful on this topic: http://blogs.msdn.com/dgorti/.

    Bill


    • Marked as answer by Jason Horn Monday, October 27, 2008 3:17 PM
    Thursday, October 23, 2008 5:33 PM
    Moderator

All replies

  • One question and one suggestion to get controller logging working:

    * Do any of the three machines have more than one network card enabled?   If so, which machines?    There is some configuration you need to do in a config file for the controller or agent in this case or in the registry if the Dev Box has multiple network cards are enabled.    (If this is the case, I'll send details on how to configure this.)

    * The problem with the controller logging is probably that the user running the controller service doesn't have permission to write to the default location for vsttcontroller.log which is under Program Files.     To workaround this, set CreateTraceListener to "no" and instead add a trace listener in the system.diagnostics section that points to a path to which the controller user can write (the user's Documents and Settings folder will work).    For example, I repro'd your error starting the controller with logging enabled by running the controller service as user "eqtbuild", then I set "CreateTraceListener" to "no" and add the following section to get the vsttcontroller log in the Documents and Settings folder:

      <system.diagnostics>
        <trace autoflush="true" indentsize="4">
            <listeners>
               <add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="D:\Documents and Settings\eqtbuild\vsttcontroller.log" />
            </listeners>
        </trace>
        <switches>
            <!-- You must use integral values for "value".  Use 0 for off, 1 for

                 error, 2 for warn, 3 for info, and 4 for verbose. -->
            <add name="EqtTraceLevel" value="4" />
        </switches>

    Tuesday, October 21, 2008 5:29 PM
    Moderator
  •  The verbose logging is working now, thanks!  I have only 1 NIC enabled on the dev box, the others only have a single Virtual NIC installed.  here is the exception I am getting:  The full log is pretty big.  Let me know what / if I can provide more...  Thanks again for your help!

    StateMachine(RunState): Exception while handling state Deploying: System.Security.Authentication.InvalidCredentialException: The server has rejected the client credentials. ---> System.ComponentModel.Win32Exception: The logon attempt failed
       --- End of inner exception stack trace ---

    Server stack trace:
       at System.Net.Security.NegoState.ProcessAuthentication(LazyAsyncResult lazyResult)
       at System.Net.Security.NegotiateStream.AuthenticateAsClient(NetworkCredential credential, String targetName, ProtectionLevel requiredProtectionLevel, TokenImpersonationLevel allowedImpersonationLevel)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.CreateAuthenticatedStream(Stream netStream, String machinePortAndSid)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.CreateSocketHandler(Socket socket, SocketCache socketCache, String machinePortAndSid)
       at System.Runtime.Remoting.Channels.SocketCache.CreateSocketHandler(Socket socket, String machineAndPort)
       at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket(EndPoint ipEndPoint)
       at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket()
       at System.Runtime.Remoting.Channels.RemoteConnection.GetSocket()
       at System.Runtime.Remoting.Channels.SocketCache.GetSocket(String machinePortAndSid, Boolean openNew)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.SendRequestWithRetry(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
       at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at Microsoft.VisualStudio.TestTools.Common.FileCopyService.get_DeploymentFlags()
       at Microsoft.VisualStudio.TestTools.Controller.ControllerExecution.DoDeployment()
       at Microsoft.VisualStudio.TestTools.Controller.ControllerExecution.RunStateDeploying()
       at Microsoft.VisualStudio.TestTools.Common.StateMachine`1.Execute()
    [E, 976, 12, 2008/10/21 16:35:03.432] QTController.exe: ControllerSpooledJob.AddMessage: Failed calling event listener Microsoft.VisualStudio.TestTools.Controller.SecureEventContainer: System.Security.Authentication.InvalidCredentialException: The server has rejected the client credentials. ---> System.ComponentModel.Win32Exception: The logon attempt failed
       --- End of inner exception stack trace ---

    Server stack trace:
       at System.Net.Security.NegoState.ProcessAuthentication(LazyAsyncResult lazyResult)
       at System.Net.Security.NegotiateStream.AuthenticateAsClient(NetworkCredential credential, String targetName, ProtectionLevel requiredProtectionLevel, TokenImpersonationLevel allowedImpersonationLevel)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.CreateAuthenticatedStream(Stream netStream, String machinePortAndSid)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.CreateSocketHandler(Socket socket, SocketCache socketCache, String machinePortAndSid)
       at System.Runtime.Remoting.Channels.SocketCache.CreateSocketHandler(Socket socket, String machineAndPort)
       at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket(EndPoint ipEndPoint)
       at System.Runtime.Remoting.Channels.RemoteConnection.CreateNewSocket()
       at System.Runtime.Remoting.Channels.RemoteConnection.GetSocket()
       at System.Runtime.Remoting.Channels.SocketCache.GetSocket(String machinePortAndSid, Boolean openNew)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.SendRequestWithRetry(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream)
       at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
       at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at Microsoft.VisualStudio.TestTools.Controller.IControllerEvents.ControllerEvent(TestMessage report)
       at Microsoft.VisualStudio.TestTools.Controller.ControllerSpooledJob.AddMessage(TestMessage message)

    • Edited by Jason Horn Tuesday, October 21, 2008 8:42 PM did answer all Bill's questions
    Tuesday, October 21, 2008 8:40 PM
  • It looks like a remote login credentials problem such as:

    1. The user running the test must be a member of the "TeamTestControllerUsers" group defined on the controller
    2. There could be some permissions issue between the controller service and the agent service, but I'm pretty sure in that case you would not see the Agent show up as Ready in the Administer Test Controllers dialog.

    Can you describe the user accounts for the user running the test, the controller and agent, and any other details about the security configuration of these machines that might be relevant.     I may need to get one or two other people to take a look at this, but they'll probably want at least that info and maybe more.

    Bill

    Wednesday, October 22, 2008 6:26 PM
    Moderator
  • Current configuration, let me know if you need more, Thanks, as always, for your continued help:

    Dev Box
        Administrators group:
        My domain account - Running VS
    Controller Box (Visual Studio Team Test Controller service running under "VSTS_Controller" account)
        Administrators group:
            My Domain Account
            VSTS_Setup
        Users group:
            VSTS_Agent
            VSTS_Controller
            VSTS_Setup
        TeamTestControllerUsers group:
            My Domain Account
            VSTS_Setup
            VSTS_Controller
        TeamTestControllerAdmins group:
            My Domain Account
            VSTS_Setup
            VSTS_Controller
        TeamTestAgentService group:
            VSTS_Agent
        Performance Log Users group:
            My Domain Account
            VSTS_Setup
            VSTS_Controller
        Performance Monitor Users group:
            My Domain Account
            VSTS_Setup
            VSTS_Controller
    Agent Box (Visual Studio Team Test Agent service running under "VSTS_Agent" account)
        Administrators group:
            My Domain Account
            VSTS_Setup
        Users group:
            VSTS_Agent
            VSTS_Controller
            VSTS_Setup
        Performance Log Users group:
            My Domain Account
            VSTS_Setup
            VSTS_Controller
            VSTS_Agent
        Performance Monitor Users group:
            My Domain Account
            VSTS_Setup
            VSTS_Controller
            VSTS_Agent
    System Under Test (Web App Server)
        Administrators group:
            My Domain Account
        Users group
            VSTS_Controller
        Performance Log Users group:
            My Domain Account
            VSTS_Controller
        Performance Monitor Users group:
            My Domain Account
            VSTS_Controller


    Wednesday, October 22, 2008 7:47 PM
  • I think the problem may be that the controller service (running as VSTS_CONTROLLER) actually needs to call back over .NET remoting to the dev box in order to copy files that need to be deployed, and that user has no permission to login to your dev box.   I'm pretty sure that if you ran both the controller and agent services as your domain account everything would work.    I suspect you don't want to do that for security reasons, but it might be worth trying as an experiment, and if that works then give the VSTS_CONTROLLER an account on your dev box (you may need to do the same for the VSTS_Agent user on the controller). 

    Also, you may find Durga Gorti's blog helpful on this topic: http://blogs.msdn.com/dgorti/.

    Bill


    • Marked as answer by Jason Horn Monday, October 27, 2008 3:17 PM
    Thursday, October 23, 2008 5:33 PM
    Moderator
  • Got past "Pending"!!!  Thanks for all your help!
    Monday, October 27, 2008 3:17 PM
  • I know this is an old post, but I have a very similiar configuration and problem.  Any chance you know what you did to get past this?


    moke

    Friday, December 7, 2012 3:17 PM