Thursday, August 25, 2011

Comprehensive SVN Mirror

Subversion is critical for any software development company. Most importantly you might want to have separate mirror in local premises and other one is public place.
Also users should not be able to detect whether either he is using local mirror or public repository. Unless it become annoying. You can redirect users to local mirror using local DNS .

Assume your public SVN is svn.example.com and mirror is svnslave.example.com. svnslave.example.com domain name will only be used to do the mirroring among servers. Users will be using svn.example.com accessing the svn independent of the location.

Following picture will give you a brief idea about how the replication taken place.



svnslave will redirect commits to master. Master will replicate the slave realtime.

Configurations on Main repo - svn.example.com

Change post-commit and post-revprop-change scripts as follows to replicate all commits and changes to slave.

post-commit

#!/bin/sh
/usr/bin/svnsync --source-username=svnsync --sync-username=svnsync sync https://svnslave.example.com/test2/test/proxy-sync/ 2>>/var/log/svnsynclog &


post-revprop-change

#!/bin/sh
REV=$2
/usr/bin/svnsync --source-username=svnsync --sync-username=svnsync https://svnslave.example.com/test2/test/proxy-sync/ $REV 2>>/var/log/svnsynclog &

Configurations on Apache


svn.example.com virtual host should be as follows. create svnsync user using following command.

htpasswd -c /etc/apache2/allowed svnsync


---------------- Start Vhost config -----------------------------



ServerName svn.example.com

SSLProxyEngine on
SSLEngine On

SSLCertificateFile /etc/apache2/example.crt
SSLCertificateKeyFile /etc/apache2/example.key
SSLCertificateChainFile /etc/apache2/chain.crt


DAV svn
SVNPath /home/svn/test
AuthType Basic
AuthName "test master SVN Login"
AuthUserFile /etc/apache2/allowed
SSLRequireSSL

Require valid-user







--------- End Vhost Config -------------------------------------------





Configurations on mirror - svnslave.example.com

Install apache subversion and apache-svn module modules as follows.


apt-get install apache2 subversion libapache2-svn


enable proxy module and ssl. Slave will redirect commit to master repository using SVNMasterURI directive. It need to have proxy modules enabled.


a2enmod proxy
a2enmod proxy_http
a2enmod ssl


Create a slave svn repository.

svnadmin create --fs-type=fsfs /home/svn/test


Change pre-revprop-change as follows to makesure syncing user will alwaylibapache2-svns be svnsync

#!/bin/sh
USER=$3
if [ "$USER" != "svnsync" ]; then
echo >&2 "Only the svnsync user is allowed to change revprops"
exit 1
fi
exit 0




svnslave.example.com virtual host should be as follows. create svnsync user as
"htpasswd -c /etc/apache2/allowed svnsync" and configure a strong password.

----------- Start Vhost Config ---------------------------------------




ServerName svnslave.example.com
ServerAlias svn.example.com

# The alias directive used due to local
#users will be refering as svn.example.com

SSLProxyEngine on
SSLEngine On

SSLCertificateFile /etc/apache2/example.crt
SSLCertificateKeyFile /etc/apache2/example.key
SSLCertificateChainFile /etc/apache2/chain.crt


DAV svn
SVNPath /home/svn/test
SVNMasterURI https://svn.example.com/test/test
AuthType Basic
AuthName "test slave SVN Login"
AuthUserFile /etc/apache2/allowed

SSLRequireSSL

Require valid-user





# svnsync will not allow if SVNMasterURI directive also there. Therefore
# you need to have seperate location match to have svnsync as follows.


DAV svn
SVNPath /home/svn/test
Order deny,allow
Deny from all
Allow from svn.example.com





-----------End Vhost Config---------------------------------



Now you can initialize slave as follows at the svn.example.com.

svnsync --source-username=svnsync --sync-username=svnsync init https://svnslave.example.com/test/test/ https://svn.example.com/test/test/


Following command will give you the stat of salve. It should return the master repo url.

svn propget svn:sync-from-url --revprop -r 0 https://svnslave.example.com/test/test


Now start syncing the master to slave.

svnsync --source-username=svnsync --sync-username=svnsync sync https://svnslave.example.com/test/test


If you have a huge master repo, some times slave locks when running the svnsync. You have to remove the lock as follows in which cases.

svn pdel --revprop -r 0 svn:sync-lock file:///home/svn/test

4 comments:

  1. Great article, thank you. One minor issue: in the Apache configs there are spurious characters ="" in some lines.

    ReplyDelete
    Replies
    1. MM problem with the syntax highlighter

      Delete
  2. If you configure the replication for an enterprise that has an Active Directory, then take a closer look at VisualSVN Server's Multisite Repository Replication feature.

    It allows slaves to be (!) writable, it is faster and much more stable than a write-through proxy with `svnsync`. Replicates access permissions as well.

    ReplyDelete