none
Merging Theory Question

    Question

  • I will describe as best I can.
    Assume the following branch architecture:
    Production - Build - Integration - Dev

    Production matches what is currently in production and released to end users.
    Build is the testing branch
    Integration could be considered the "Trunk" branch where project branches
    Dev is where maintence items are worked on

    For example:
    Before I merge from Dev to Integration I try to clean up candidate changesets going from Integration into Dev.
    Then when I merge changesets from Dev to Integration I have a host of new candidate changesets to merge back into Dev. I kind of get that in theory I have created "new" files but it seems strange that a "merge" changeset would be seen as a changeset to go back to the branch it just came from.

    My question:
    If I clean up these new candidate changesets am I in essense starting an endless cycle of changesets to merge back and forth? Is there a way to prevent this from happening? Is this normal?

    Thanks,
    Ross
    Friday, September 18, 2009 4:27 PM

Answers

  • This happens when the Integration->Dev merge has conflicts which are resolved as AcceptMerge ("auto merge / merge changes in merge tool") or AcceptYours ("keep target").  Only changes marked AcceptTheirs ("copy from source branch") can be safely excluded from future merges in the opposite direction.

    Note: when the merge is actually submitted, the file contents you upload can override previous calls you made (or didn't make) to Resolve.  For example, if you chose one of the AcceptMerge options but the resulting contents was identical to the source, the server will treat it as an AcceptTheirs.  By the same token, if you modify a file that was handled w/o conflict (source-only changes are implicitly AcceptTheirs) while it's in the pending "merge, edit" state, it will be treated as AcceptMerge upon checkin.

    Source control tools like TFS do their best to maintain the integrity merge history + the safety of downstream operations.  But they can't do it without help -- especially if you want to minimize warnings/conflicts/etc at the same time (a rather contradictory goal if you think about it).  The best guidelines I've come across are from Perforce's Laura Wingerd.  "merge down, copy up"  http://video.google.com/videoplay?docid=-577744660535947210&ei=BAu0SqXsD5bWrQLz7Zj_AQ&q=laura+wingerd&hl=en 
    • Marked as answer by ssorname Monday, September 21, 2009 5:42 PM
    Friday, September 18, 2009 10:42 PM
    Moderator
  • I don't think you need to change your whole process.  For small teams, a simple branch structure like the one you described is sufficient; less is more.  See here for some excellent tips on running a typical dev->test->production model like yours: http://tfsbranchingguideii.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=20785#ReleaseFiles 

    Anyway, there are many important ideas raised in her Powerpoint.  The one that plays a crucial role here is "merge down, copy up."  For you, that means it's entirely appropriate to resolve conflicts in a variety of ways (AcceptTheirs, Merge, or Yours; auto or manual) when going from Integration to Dev.  However, you should strive for the subsequent Dev->Integration push to always be a pure AcceptTheirs.  Ideally there are no conflicts and everything is implicitly pended as AcceptTheirs from the start.  If there are conflicts, you want to have the confidence to blindly choose "copy from source branch" because -- thanks to the rules of the road -- you know the softer Dev branch has already absorbed + stabilized everything it's pushing toward the harder branches.  Upholding this principle is the real reason why merging in the other direction immediately beforehand is a good practice; it's not just about minimizing the # of times you have to click AutoMerge.

    If there are multiple Dev teams sharing the Integration branch, you may need a locking mechanism between the "merge down" (aka forward integration or FI) and the "copy up" (RI).  Ideally you have been FI'ing early and often, the time gap during the final sequence can be as small as possible.  Even so, an organization large enough to worry about problems at this scale will by definition be large enough for branch lockdowns to cause pain.  Unfortunate equation: more people sharing a branch (ie the closer it is to the trunk) = more RI-related lock contention there will be = more people will be impacted each time.  Suffice to say, following the "rules" gets exponentially more important.

    However, locking isn't strictly necessary.  Merges are performed in workspaces, which already have their own mechanism for isolation and conflict resolution.  Namely: nothing says you have to run Get Latest before merging to Integration.  You can sync to the time of the FI, run the RI, and blindly resolve everything AcceptTheirs without worrying about stomping over unrelated changes.  Save this work in a shelveset, then Get Latest -- now you can resolve any conflicts between your final stabilization work and other simultaneous changes to Integration in a separate step from when you resolved the inter-branch conflicts.  Disadvantage: you don't get to commit the actual RI work as its own changeset.  As you Get the overlapping changes into the workspace and start to resolve any conflicts, the entire validation process must be done locally; without an intermediate checkin, Team Build & any automated tests you've attached to it cannot help you.  Once everything's finally merged together (possibly including brand new code that reacts to those changes) and checked in, the version history can look opaque.**  You'll never be able to pinpoint one row in the History dialog or Annotate tool and say "aha, those are the changes RI'd by the Foobar feature."  Impossible to say up front how trivial or vital that capability might be in the future.  The pain that locking can cause is both more obvious & hurtful, but at least you can put strict bounds on it (ie, once unlocked you're back to happy branch users and pristine history).

    **And not just for humans!  This scenario is a good illustration why TFS has to treat AcceptMerge resolutions so carefully in the general case that it feels annoyingly naive in simpler cases.  Sometimes an AcceptMerge really does involve uploading "brand new code" or a mishmash thereof, despite the original intent being a purely "copy up" RI from Dev.
    • Marked as answer by ssorname Monday, September 21, 2009 5:42 PM
    Saturday, September 19, 2009 3:54 AM
    Moderator

All replies

  • This happens when the Integration->Dev merge has conflicts which are resolved as AcceptMerge ("auto merge / merge changes in merge tool") or AcceptYours ("keep target").  Only changes marked AcceptTheirs ("copy from source branch") can be safely excluded from future merges in the opposite direction.

    Note: when the merge is actually submitted, the file contents you upload can override previous calls you made (or didn't make) to Resolve.  For example, if you chose one of the AcceptMerge options but the resulting contents was identical to the source, the server will treat it as an AcceptTheirs.  By the same token, if you modify a file that was handled w/o conflict (source-only changes are implicitly AcceptTheirs) while it's in the pending "merge, edit" state, it will be treated as AcceptMerge upon checkin.

    Source control tools like TFS do their best to maintain the integrity merge history + the safety of downstream operations.  But they can't do it without help -- especially if you want to minimize warnings/conflicts/etc at the same time (a rather contradictory goal if you think about it).  The best guidelines I've come across are from Perforce's Laura Wingerd.  "merge down, copy up"  http://video.google.com/videoplay?docid=-577744660535947210&ei=BAu0SqXsD5bWrQLz7Zj_AQ&q=laura+wingerd&hl=en 
    • Marked as answer by ssorname Monday, September 21, 2009 5:42 PM
    Friday, September 18, 2009 10:42 PM
    Moderator
  • I watched the video. Very interesting. I will have to work out how that fits with our team.
    We don't really have a "Release 1" or "Release 2" we are pretty much working on a strictly internal application so we do frequent releases of new code, bug fixes and new development.

    Ross
    Saturday, September 19, 2009 1:23 AM
  • I don't think you need to change your whole process.  For small teams, a simple branch structure like the one you described is sufficient; less is more.  See here for some excellent tips on running a typical dev->test->production model like yours: http://tfsbranchingguideii.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=20785#ReleaseFiles 

    Anyway, there are many important ideas raised in her Powerpoint.  The one that plays a crucial role here is "merge down, copy up."  For you, that means it's entirely appropriate to resolve conflicts in a variety of ways (AcceptTheirs, Merge, or Yours; auto or manual) when going from Integration to Dev.  However, you should strive for the subsequent Dev->Integration push to always be a pure AcceptTheirs.  Ideally there are no conflicts and everything is implicitly pended as AcceptTheirs from the start.  If there are conflicts, you want to have the confidence to blindly choose "copy from source branch" because -- thanks to the rules of the road -- you know the softer Dev branch has already absorbed + stabilized everything it's pushing toward the harder branches.  Upholding this principle is the real reason why merging in the other direction immediately beforehand is a good practice; it's not just about minimizing the # of times you have to click AutoMerge.

    If there are multiple Dev teams sharing the Integration branch, you may need a locking mechanism between the "merge down" (aka forward integration or FI) and the "copy up" (RI).  Ideally you have been FI'ing early and often, the time gap during the final sequence can be as small as possible.  Even so, an organization large enough to worry about problems at this scale will by definition be large enough for branch lockdowns to cause pain.  Unfortunate equation: more people sharing a branch (ie the closer it is to the trunk) = more RI-related lock contention there will be = more people will be impacted each time.  Suffice to say, following the "rules" gets exponentially more important.

    However, locking isn't strictly necessary.  Merges are performed in workspaces, which already have their own mechanism for isolation and conflict resolution.  Namely: nothing says you have to run Get Latest before merging to Integration.  You can sync to the time of the FI, run the RI, and blindly resolve everything AcceptTheirs without worrying about stomping over unrelated changes.  Save this work in a shelveset, then Get Latest -- now you can resolve any conflicts between your final stabilization work and other simultaneous changes to Integration in a separate step from when you resolved the inter-branch conflicts.  Disadvantage: you don't get to commit the actual RI work as its own changeset.  As you Get the overlapping changes into the workspace and start to resolve any conflicts, the entire validation process must be done locally; without an intermediate checkin, Team Build & any automated tests you've attached to it cannot help you.  Once everything's finally merged together (possibly including brand new code that reacts to those changes) and checked in, the version history can look opaque.**  You'll never be able to pinpoint one row in the History dialog or Annotate tool and say "aha, those are the changes RI'd by the Foobar feature."  Impossible to say up front how trivial or vital that capability might be in the future.  The pain that locking can cause is both more obvious & hurtful, but at least you can put strict bounds on it (ie, once unlocked you're back to happy branch users and pristine history).

    **And not just for humans!  This scenario is a good illustration why TFS has to treat AcceptMerge resolutions so carefully in the general case that it feels annoyingly naive in simpler cases.  Sometimes an AcceptMerge really does involve uploading "brand new code" or a mishmash thereof, despite the original intent being a purely "copy up" RI from Dev.
    • Marked as answer by ssorname Monday, September 21, 2009 5:42 PM
    Saturday, September 19, 2009 3:54 AM
    Moderator
  • That does make sense. The one catch is that we do not merge everything from Dev into Integration. We only merge the changes for work that is "completed" not in progress. So I can't take the latest from Dev only specific changesets. I am starting to think that Integration is our baseline.
    One issue we face is that our developers check code in before it is reviewed. When we first started using TFS we would review shelfsets but we found it difficult to review shelfsets for a couple of reasons. 1) If code is rejected from a review there isn't a good way to review changes made to a shelfset. 2) They wanted to be able to create a shelfset, undo pending changes and then when review is done checkin that changeset. I know good practice is to not check in until it complete but I am getting push back changing that rule. I think that will make our code base a lot safer if I can get that changed.

    I will read the document you linked to.

    Thank You for your responses.

    Ross
    Sunday, September 20, 2009 9:38 PM
  • I should say we do not have multiple dev teams sharing Integration. The thinking behind was a place to integrate the changes that from Dev and Build (bug fixes during testing). It also serves as the baseline for all ProjectX branches so we do not face the scenario where we have to accept changes not ready for release in Dev.
    This is also why I am thinking that Integration really is the Mainline for our enviornment.

    I will keep reading.

    Ross
    Sunday, September 20, 2009 9:50 PM