version control workflow for environment with shared code

Question

I work at a company that is looking into "upgrading" our version control system. We have about 200 websites that operate in their own directories. However, all of these websites use the same shared code. We basically use the 200 directories for the display code (html/css) and then the shared code for the processing and database queries. Its a little more detailed but I'll leave it at that for now. We also have a backend system that primarily uses shared code only.

We currently have multiple programmers and multiple coders/designers that all need access to the code on a regular basis. Unfortunately our shared code isn't exactly modular, so it is hard to split something off into a "project" when something needs done since so many files are hit. I am basically looking for suggestions on the best workflow available for a version control system. As far as we can tell from research, git seems to be the way to go. But being the version control newbies that we are, we still have concerns on general workflow. Can anyone give me any advice on this topic?

Edit: I'm trying to be a little more detailed, hopefully I answer some questions from the comments. Pretty much anyone is allowed to change any code at any time. If a change is made to shared code then it is immediately applied to the central code base. We don't really have a versioning scheme. We used to in the past but it was never enforced when new developers were brought on board. The shared code is clearly separated from the rest of the code. There are a few files in our system that get edited a lot. We are constantly "stepping on each other's toes" with these files. These files are very important to our system. I'll be the first one to say this is not an ideal work situation. We are well aware these files need to go. We are in the planning process of modularizing them but we would still need to consider them when choosing a versioning system. I guess another good thing to mention is the fact that we have a large number of files. This isn't ideal either but its happened primarily from not having a sound version control system put in place. Many developers over the years would just save a back up of a file if making a significant change. This is another reason we want to move away from the current scheme. Thanks for your responses already.

Answer 1

@jbc1785: I'm going to take a stab at this based on the following assumptions about your code and your team -

1. You currently do not have any code in a version control system and you are new to version control
2. Every developer can access and change any file in the code base
3. You have multiple developers working on multiple projects in parallel
4. The code is not very modular and there are some key files in your code base that need to be touched almost every time you make any change.

In your situation, I would recommend that you start off with a centralized version control system like Subversion (SVN) instead of a distributed one like Git. I have worked in places without version control systems and (as you have correctly identified) it is vital to put in place a version control tool that simple and provides a simple workflow that teaches good version control habits. Since you have a system where you have a high chance of multiple developers making changes to the same file at the same time, you should leverage a centralized tool like subversion better to reduce the amount of errors and confusion that can occur with merge conflicts.

Subversion provides a "lock" functionality that allows users to lock files that they are editing - I recommend you use this in the initial stages so that your team gets used to the concepts of checking out and editing files. The lock status will reduce the initial confusion by making clear that a file is being currently worked on by someone and that conflicts will occur if the file is changed by someone else.

As the team gets used to the concepts of version control you can disable the "lock" functionality and introduce the concept of merging files, "merge conflicts" and their resolution. Usually once a team gets comfortable with the concept of locked files, the necessary caution and communication needed for working on files in parallel gets developed.

Once the team gets used to working with Subversion, a DVCS like Git is the logical next step. It's very easy (there are multiple tools) to move from Subversion over to Git. In fact you can run Subversion and Git in parallel and migrate over. Git scores over Subversion in terms of performance, ability to scale and the fact that since it's distributed there are no single points of failure. I think the only problem with Git is that it's command syntax is a bit cryptic and it takes a bit of getting used to.

Update: Since the comments for this post are getting numerous and hidden, I thought I'd add them to the post itself in an effort to make it more easily seen.

According to @Tamás Szelei and @JanHudec, DVCS is the way to go since locking is a bad feature of centralized version control systems that should be avoided. The contention is that Subversion dont have a good merge feature and therefore uses locking as a way to get around that deficiency.

While I accept that merging in Subversion is not as great as it is in a DVCS like Git, I disagree that using locks and the workflow encouraged by Subversion is necessarily an evil thing that should be avoided at all costs. I think Subversion has its place and in certain situations (like in case of binary files - mentioned in the other answer to this question) might even be better suited than a DVCS.

This is a matter of opinion and I still believe that given the situation mentioned in the question (listed in the beginning of this answer) using Subversion is a good way to instill version control practices and workflows in a team that have never used version control before.

Answer 2

First the workflow:

1. You will need a server that will act as the central storage for the code and install appropriate system there (more on choice later). It will have to be accessible by everybody who need to modify code and for security reasons should be separate from your regular web server. Unless you really need to make it visible from the internet, don't.

2. On the web server, make the data directories checkouts from the version control system. There will be one checkout for each site and than one or multiple checkouts of the shared code. If you currently have the shared code on the server just once, you can go with one checkout now and switch to per-site checkouts when you get into situation that some site will need new code and another will need to stay on the old one.

   Well, if the sites are simple or very closely related anyway, you can just have everything in one big checkout and one big working tree.

3. Arrange for some mechanism to update the web server when changes are pushed to the version control. You can have either manual or automatic process or combination thereof. I would suggest automatic process for updating the test environment (so you can review your changes immediately after pushing them in) and manual for updating the live one (so only reviewed changes get published).

   In either case you'll have a script on the web server, that will run appropriate update command on all the checkouts. The script shall be started either by the web server (by accessing special URL) or by ssh trigger (special ssh key bound to run the specific command only). Than you can start it manually or from "post-commit" hook.

4. Everybody who will be making changes will check the parts they need to work on on their local computers. The layout will be the same as on the server, but if they never touch some site, they don't need to check it out.

Now to make a change, you'll run appropriate update command on your local checkout, edit the files, test with your locally-installed web server if possible and commit and push to the central server. Trigger the update script on the test server if it's not done automatically and have the changes reviewed. When they are reviewed, you or the tester will trigger the update script on the live server, preferably explicitly telling it up to which version to update (the one reviewed).

The process should work as much the same as it does now except you'll use the version control system to deploy the files instead of whatever copying you are doing now.

"stepping on each other's toes":

Version control systems generally work with the "edit-merge-commit" cycle these days. That is you start editing the file and when you are satisfied with your modifications, you run a command to merge them with whatever other changes happened meanwhile and than commit the result.

The merge is a line-based text operation. It looks at what lines were added and removed (modify = remove + add) on each side and if the changes are to different parts of the file, it will apply both. If both modified the same part of the file, there is a conflict, which can be resolved using special editor that shows you all the versions and allows you to pick the version you want. There are e.g. kdiff3, meld and many other.

This simple approach works rather well for most kinds of files, but you should avoid unnecessary reformatting as it may cause conflicts. It however does not work for binary files, which includes images and video. There the version control tool will simply say the whole file is a conflict and you'll have to pick one or other version.

So if you happen to modify them a lot, you may want to use the "lock-edit-commit" cycle instead, i.e. the person wanting to modify such file will mark it as being edited and others will be told that they shouldn't touch it now. This is however only supported in centralized system. However in my experience those files are rarely modified by more than 1 person or with any significant frequency, so the very rare case of conflict simply isn't worth the extra work using the locks.

Choosing version control system:

The most modern and more flexible these days is distributed version control systems and of those the two most popular are git and mercurial. There are already some links to comparisons in comments, so I won't repeat them.

However if you happen to need locking, you'll have to use a centralized version control system. The most common option there is subversion. However it's branching support is really poor compared to git and mercurial and it's quite a bit slower. So only use if you really edit binary files a lot.

Note: Now I remember I worked with something that really needed the locking approach -- Adobe CreativeSuite 3 sources for Flash. I believe it's no longer such problem with the new tools, but we couldn't use that than, because it didn't support the Flash Lite target.

Answer 3

when we migrated from SVN to GIT we have based our repository usage model on gitflow - http://nvie.com/posts/a-successful-git-branching-model/

This branching model really helped us in our situation of 5 million lines of copy-paste legacy nightmare code and 3 scrum teams keeping application alive in 1 week iterations

We also use gerrit - http://code.google.com/p/gerrit/ for code review and access control, but it's not perfect tool for beginners (it adds a lot of complexity in day-to-day usage of your repositories, though it prohibits non administrators from breaking anything)