the bus blog

As I said in this post I’ve been exploring and using Subverison a lot recently, and with plans to start a hosted subversion arm to the web hosting business I need to keep working on the web based management tool. As everyone knows backups are crucial, so the first step towards my web based subversion management tool is the backup side of things. This is the first version of the script, I’ve been running it nightly for about 2 weeks now and it’s behaved very well so far.

I started with the script provided by Mark Shead (Backing up Subverison Automatically) however with plans for expansion I didn’t want to end up with a backup script for each repository. In addition to this I wanted the script to integrate with my envisaged web based SVN management tool, that meant database integration. In setting up the above script I found this article very helpful.

I ran with several versions of Marks script all running via cron and sending the output as email for a few days, then decided that now was a good time to integrate the script to a database and backup all of the repositories in the database. For a guide to using the script via cron see the end of the article.

I would include a SQL dump from PHPMyAdmin - but wordpress won’t let me include that in the post -but you need a single table called repo with three fields (repo_id, repo_location and repo_prefix). Any questions or comments leave a comment or email me.

Watch out for the mis-spelled SE!!!L!!!ECT - wordpress also won’t let me save the post with that line (or this line!) for some reason. I’m looking into why.

#perl modules to use
use DBI;
use DBD::mysql

#DBI config values
$host = "localhost";
$database = "database_name";
$user = "user_name";
$password = "password";

#Data Source Name
$dsn = "dbi:mysql:$database:localhost:3306";

#Perl DBI Connect
$connect = DBI->connect($dsn, $user, $password) or die "Unable to connect: $DBI::errstr\n";

#write the query
#Perl DBI Connect
$connect = DBI->connect($dsn, $user, $password) or die "Unable to connect: $DBI::errstr\n";

#write the query
$query = "SE!!!Insert L!!!ECT repo_location, repo_prefix FROM repo";
$query_handle = $connect->prepare($query);

#run the query
$query_handle->execute();

$query_handle->bind_columns(undef, \$repo_location, \$repo_prefix);

while($query_handle->fetch()) {

#my $svn_repo = $repo_location;
my $bkup_dir = "/subversion/backup";
#my $bkup_file = $repo_prefix;
my $tmp_dir = "/subversion/tmp";
my $bkup_svr = "backup.green-host.co.uk";
my $bkup_svr_login = "root";
my $remote_path = "/home/green/svn_backup";

print "\n-------------\n";
print "Backing up Repository $repo_location";
print "\n-------------\n";

$bkup_file = $repo_prefix . `date -Is`;
chomp $bkup_file;
my $youngest = `svnlook youngest $repo_location`;
chomp $youngest;

my $dump_command = "svnadmin -q dump $repo_location > $bkup_dir/$bkup_file ";
print "\nDumping Subversion repo $repo_location to $bkup_file...\n";
print `$dump_command`;
print "Backing up through revision $youngest... \n";
print "\nCompressing dump file...\n";
print `gzip -9 $bkup_dir/$bkup_file\n`;
chomp $bkup_file;
my $zipped_file = $bkup_dir . "/" . $bkup_file . ".gz";
print "\nCreated $zipped_file\n";
print `scp $zipped_file $bkup_svr_login\@$bkup_svr:$remote_path`;
print "\n$bkup_file.gz transfered to $bkup_svr\n";

#Test Backup
print "\n-------------\n";
print "Testing Backup";
print "\n-------------\n";
print "Downloading $bkup_file.gz from $bkup_svr\n";
print `scp $bkup_svr_login\@$bkup_svr:$remote_path/$bkup_file.gz $tmp_dir/`;
print "Unzipping $bkup_file.gz\n";
print `gunzip $tmp_dir/$bkup_file.gz`;
print "Creating test repository\n";
print `svnadmin create $tmp_dir/test_repo`;
print "Loading repository\n";
print `svnadmin -q load $tmp_dir/test_repo < $tmp_dir/$bkup_file`;
print "Checking out repository\n";
print `svn -q co file://$tmp_dir/test_repo $tmp_dir/test_checkout`;
print "Cleaning up\n";
print `rm -f $tmp_dir/$bkup_file`;
print `rm -rf $tmp_dir/test_checkout`;
print `rm -rf $tmp_dir/test_repo`;
}

In one of my biggest online gaffs to-date, when I moved this blog from http://craddy.swdl.org to http://swdl.org I forgot about redirecting RSS feeds. That’s hopefully now been fixed - so if you’re reading this as a first post for a very very long time then that’s why!

If anyone’s interested this is how you can redirect a whole domain / sub-domain:


Options +FollowSymLinks
RewriteEngine on

RewriteCond %{HTTP_HOST} !^www.swdl.org [NC]
RewriteRule ^(.*)$ http://www.swdl.org/$1 [R=301,L]

Obviosly change swdl.org to the domain you want to re-direct to and place the above in a .htaccess file in the source domain / sub-domain.

I’ve been meaning to write about version control for a while, its seen to be an incredibly boring and geeky subject. I started using it for my dissertation project in my final year - and now I’m using it for almost every document, script or spreadsheet I have. I think that not oly can anyone use subversion, most people should use it!

I like using it so much I’ve started to get interested in the stuff behind the scenes, the stuff that makes it work - and how that can be used to improve it. So much so I’ve setup a subversion server with plans to start selling it. Free, time unlimited, trials will be available.

So, why is it so good and how can you use it?

Well, assuming you use a repository that’s hosted on a different physical machine, the first advantage is an automatic backup. That’s a plus, but not the main event - let me explain how it works. Every time you ‘check-in’ your work subversion notes the changes that were made from the previous revision and remembers that. So you can go back and see all of the previous versions - and what was changed in each version.

This really comes into its own when you figure out that last Friday you deleted a paragraph that you just realised you want to keep. You can simply view the version before that paragraph was removed and get it back.

There is a lot more to subversion that that, especially when you get more than one person working on the same thing. But I really think if you have the tiniest inkling that subversion might be interesting then give it a go!

If you fancy giving subversion a go then you’ll definitely need TortoiseSVN, if you can’t find anyone offering a free trial repository then drop me a line. I’m working on a web client for managing subversion repositories so testers are needed!

I’m trying to find a decent beginners tutorial to TortoiseSVN - but not having much luck so watch this space.