locked
TF14061: The workspace xxxx does not exist RRS feed

  • Question

  • A desperate plea for help... Periodically our TFS build agent goes into spasm and fails to perform builds. So far the only solution we have found is to remove the build agent and then re-add it. Below is an example of the output we get:

    Anecdotally this seems to often happen after Windows / TFS updates or server machine reboots.

    Any advice gratefully received (and I will pass on to the TFS admin at our location)

    Andrew.

    Tuesday, April 19, 2016 3:59 PM

All replies

  • Hi Andrew Ch._,

    Thank you for posting here.

    You could try to first login on your VSO build agent server. Find the path "C:\Users\tfsadmin\Desktop\New agent\_work\". Then delete the folders under the path.

    Pay attention that not deleting the _work folder, but delete the subfolders under it.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Wednesday, April 20, 2016 8:11 AM
    Moderator
  • Hello Tingting,

    Thanks for the reply. I have past it on to the person dealing with our TFS problem. If I get anything back from him I will add another reply here.

    Andrew

    Wednesday, April 20, 2016 9:35 AM
  • Hi,

    I'm currently dealing with this TFS issue and still haven't succeeded in fixing it, unfortunately all of our builds are failing and have been for a week and this has become a massive problem for the company. Does some sort of priority support exist where I can speak to someone? I looked at the MSDN incident report page but the links are broken (https://msdn.microsoft.com/subscriptions/bb266240).

    Heres the full description of the issue:

    Setup:

    TFS 2015 Update 2
    Visual Studio 2015 Update 2
    Windows Server 2008 R2
    Using TFBuild (vNext?) service
    Build agent is on the TFS machine

    The Problem:

    We don't exactly know what caused it, either Windows Update or when we installed TFS Update 2 (both happened roughly the same time). All builds seemed to start failing with the error:

    TF14061: The workspace ws_2_6;Project Collection Build Service does not exist.
    Prepare repository failed with exception.

    This error has never occurred prior to installing those updates 2 weeks ago. Here's some screenshots of exactly what occurs on build from a fresh build-agent: https://imgur.com/a/5Qn7i

    On a fresh build agent install the builds work perfectly, the problems occur when the TFS machine gets restarted. As shown in the images, After a machine restart the build throws a warning about failing to scorch, then gets an error about the workspace. Subsequent builds immediately fail with the workspace error.

    What I've attempted to resolve it:

    I got the impression that the workspace got corrupted in some way so I attempted to delete it. Trying to delete the workspace with "tf workspace commands" fails with the error:

    TF14061: The workspace WORKSPACENAME;Project Collection Build Service does not exist.

    I got that error on all the workspaces that failed builds.

    I finally managed to delete them all by first running "tf workspaces /remove" command, and then running the delete command.

    I then also deleted the build agent and installed a fresh one to ensure everything was stable.

    At this point "all" builds started working again. Builds used the new agent and created new workspaces for them. We ran the same build multiple times with no problems.

    We then restarted the TFS machine (which is also the build machine). And on all the builds we immediately got the error you see above and in the images.

    I noticed that the "Clean" option in the build definition repository tab has changed. Originally builds would clean the entire workspace and Get latest, now it's just syncing with the repo since Update 2. So I used the Build.Clean variable as mentioned in the help box to delete the entire source folder, but the error still appears before cleaning the workspace even happens.

    Edit:

    I had to run the tf remove command, "then" restart the machine until I could actually delete the workspaces with tf delete command.

    Running the remove command then delete doesn't work, it just comes up with the same error. The restart in-between must be doing something.

    Edit2:

    Heres a log file from the _diag folder in the VsoAgent: http://pastebin.com/fvAGd7Yw

    It highlights the error with more detail starting from line 76.



    Wednesday, April 20, 2016 1:58 PM
  • Hello Staeton,

    That was a useful response -- thanks! So, restarting the TFS machine (server and build agent all in one) looks like the culprit rather than Windows or TFS updates.

    Here is something to investigate. When one downloads and starts a Build Agent, does it start as a service with the process owner being the same process owner after the machine reboots? Just a though.

    Andrew.

    Wednesday, April 20, 2016 2:15 PM
  • OK the solution would appear to be:

    • Clean up any existing build agent and all the workspaces associated with them
    • Install a new build agent
    • Reboot the machine BEFORE DOING ANY BUILDs
    • Then it looks like one can build / reboot the TFS machine to ones heart's content.

    Moral of the story -- reboot after doing anything. Only once a reboot has been done is it safe to use.

    Andrew.

    Wednesday, April 20, 2016 3:51 PM
  • @Tingting MO

    I am not really satisfied as this has been marked as an answer. While ACs solution seems to work, this was a very big issue for us and I feel like it shouldn't be overlooked. After all the time and investigation spent on this I still have zero knowledge of what even went wrong and I worry this could easily happen again in the future.

    I would appreciate some explanation of what actually happened, or some ideas on how I could have prevented this workspace glitch.

    Thursday, April 21, 2016 8:30 AM
  • Was the agent running as a different user before/after reboot? Do you run the agent interactively from the command prompt or as a Windows service?
    Tuesday, August 16, 2016 7:22 PM
  • Do you have the call stack? Try running a build with a variable System.Debug set to True. If the call stack doesn't appear in the build log, you may need to check the worker diagnostic log (under _diag folder of the agent directory).
    Tuesday, August 16, 2016 7:26 PM
  • Hello Eric,

    Thank you for taking the time to reply.

    To answer your questions:

    1) I would assume that the agent was run as the same use before / after the reboot -- that really is under the control of how TFS manages things. My guess is that it uses the same user each time?

    2) We have progressed now and so cannot provide the call stack from the failure.

    We seem to have been able to have fixed the problem. After installing the build agent we now make sure that the machine is rebooted before we do any builds with it. That way it seems to be resilient to future reboots.

    Andrew.

    Wednesday, August 17, 2016 7:26 AM