CMS updates into subversion -> review process

Question

We manage a range of client sites built in Wordpress and Joomla and these require regular updates to the core CMS and extensions. We keep these sites in subversion and place updates in subversion. We try to use a single revision for this.

We face some difficulties in making this process efficient, over time we would like to automate it, so we can offer the process on a fixed price basis

The process is currently as follows

1. make copy of whole folder

2. svn status |grep '^!' |sed 's/^!\s*/svn delete "/g' |sed 's/$/"/g' |sh

3. svn update

4. svn status |grep '^?' |sed 's/^?\s*/svn add "/g' |sed 's/$/"/g' |sh

5. svn ci -m "Commit message"

6. svn remove -m"temporarily remove" http://subversion.repository.com/svn/automatem/projects/client/trunk/project/foldername

Steps 5-6 are usually repeated multiple times.

What I'm looking for help with

• we are using subversion version 1.6 and 1.7, because there are no .svn folders in 1.7 in subdirectories, I wonder if the process is much easier on 1.7?

• we've added step 3 because it reduces the number of repeats in steps 5-6. However this was just trial and error and I can't quite get my head around why this is

• As I understand it, the issue in step 5-6 is that when the an extension is updated, it may delete a whole folder and then re-insert that folder with changed files. In subversion 1.6, this would remove the .svn folder, which causes a 405 Access denied error (the folder gets added, but already exists in svn). What I would need is something that inserts all .svn folders back into my working copy if the folder already exists in svn. Is there a way to accomplish this?

• Any other improvements of course appreciated.

Answer 1

This looks like a good job for externals. Joomla and Wordpress are both hosted on subversion, which makes externals a natural fit. Basically, instead of making your own copy of a folder and trying to manage the changes, you tell it this folder should be pulled from a certain revision from a certain external repository.

Answer 2

The problem you are facing is better solved using tools other than SubVersion. As a gross (mis)characterisation, SubVersion was developed around 10 years ago, with the main goal of being better than CVS because it managed directories as well as files. As a version control product its very capable, and one that I have used in the past on serious projects. However....

What you need is not version control, but change management. You need to manage the delta that a change creates in the software, but also where that change should be applied.

The free option, I'd use GIT, but if your company has the money, then Accurev is a better technical choice for this kind of problem.

With git, you would create a branch that just contains the base code for joomla, and then do each of your customer sites in seperate branches. When you are ready to pull in the current joomla version you would issue a git command such as 'git pull origin joomla' to pull in changes from branch 'joomla' Whilst this would work, the weakness is that you would need to know that an update was in joomla, and that you need to remember to pull in the changes.

With Accurev, you would create a stream that contains the joomla code, and you would tag each release. Then, in your projects stream heirachy, you would then cross link your joomla directory to the tagged revision level.

The advantages Accurev gives you over git is that you can explicitly change your cross link to any tagged version of joomla. Including rolling back a version. You have visibility of which version is currently being used, and you can easily find where each taged version is currently used. (e.g. look for the projects that are still using a old and vulnerable version of joomla). The downside is the cost of the product, the learning curve for the developers to adjust to the new way of thinking (that some will just not get it), and that the developers will inevitably grumble about the Accurev GUI.

Answer 3

Upgrading to SVN 1.7 will help alot -- it makes SVN almost bearable.

If I was stuck managing something like this in SVN and I didn't want to look at using svn externals, I would take a much different tactic here -- I would not version the core CMS with the framework at all but rather have that in a separate repo that could be pulled via svn export when needed. Upgrades involve just deploying to a new folder.

Answer 4

You forgot to mention some significant things:

• Does sites (in core or|and in modules) have customer-specific changes or they use vanilla code?
• Are clients sites also Working Copies of SVN-repos or unversioned trees?
• Why (at least for WP, can't recall Joomla state for today) you don't use embedded updater, which can update all - core, modules, themes?

Anyway, your current process doesn't have any sense (for me). In order to update in your repository external code, also served by SCM, you can have a lot less operations

WP, no local modifications

• Link your trunk (with externals) to latest tag of WP's core in WP-repository
• Link all used in installation Wordpress plugins to latest tags of these plugins in WP-plugins repository (combine all external definitions in one common location for better manageability)

When new release of anything will appear, you have only:

• Change external of this anything in order to link to new tag
• Update tree (updates will appear in your WC automagically)
• Commit changes and deploy new version of site to destination

WP, local modifications

For existing local changes workflow will be changed only in some details: your code will be in trunk, vendor's code - in some (RO) branches (also externaliz'ed), which you merge to trunk after update

Same tricks can be applied to Joomla, maybe just one-two more will be needed, when (if) Joomla will change SCM-backend from Subversion