[svnbook commit] r2941 - trunk/src/en/book
sussman
noreply at red-bean.com
Mon Dec 17 13:36:53 CST 2007
Author: sussman
Date: Mon Dec 17 13:36:51 2007
New Revision: 2941
Log:
Finish my tasks for book drop.
* src/en/book/ch06-server-configuration.xml: finish webdav-proxy stuff.
* src/en/book/ch05-repository-admin.xml: tweak output of 'svnsync sync'.
Modified:
trunk/src/en/book/ch05-repository-admin.xml
trunk/src/en/book/ch06-server-configuration.xml
Modified: trunk/src/en/book/ch05-repository-admin.xml
==============================================================================
--- trunk/src/en/book/ch05-repository-admin.xml (original)
+++ trunk/src/en/book/ch05-repository-admin.xml Mon Dec 17 13:36:51 2007
@@ -2669,17 +2669,23 @@
…
$ svnsync synchronize http://svn.example.com/svn-mirror \
--username syncuser --password syncpass
+Transmitting file data ....
Committed revision 1.
Copied properties for revision 1.
+Transmitting file data ..
Committed revision 2.
Copied properties for revision 2.
+Transmitting file data ..........
Committed revision 3.
Copied properties for revision 3.
…
+Transmitting file data .....
Committed revision 23406.
Copied properties for revision 23406.
+Transmitting file data ...
Committed revision 23407.
Copied properties for revision 23407.
+Transmitting file data ........
Committed revision 23408.
Copied properties for revision 23408.
</screen>
Modified: trunk/src/en/book/ch06-server-configuration.xml
==============================================================================
--- trunk/src/en/book/ch06-server-configuration.xml (original)
+++ trunk/src/en/book/ch06-server-configuration.xml Mon Dec 17 13:36:51 2007
@@ -144,6 +144,14 @@
</row>
<row>
+ <entry>Master-slave server replication</entry>
+ <entry>Transparent write-proxying available from slave to master</entry>
+ <entry>Can only create read-only slave servers</entry>
+ <entry>Can only create read-only slave servers</entry>
+ </row>
+
+
+ <row>
<entry>Speed</entry>
<entry>somewhat slower</entry>
<entry>somewhat faster</entry>
@@ -2551,59 +2559,251 @@
<sect3 id="svn.serverconfig.httpd.extra.writethruproxy">
<title>Write-Through Proxying</title>
- <para>intro paragraph -- present the problem of a massively
- distributed team. 80% of all traffic is read-traffic. itch
- is how to speed up remote offices. solution: write-through
- proxying makes that 80% traffic into zippy local
- accesses. Explain how single-master replication works from
- 10,000 feet: define master and slave terminology.</para>
-
- <para>requirements: at least svn 1.5 and apache 2.2.0.
- mod_proxy must be enabled. Here the walkthrough</para>
+ <para>One of the nice advantages of using Apache as a
+ Subversion server is that it can be set up for simple
+ replication. For example, suppose that your team is
+ distributed across four offices around the globe. The
+ Subversion repository can only exist in one of those
+ offices, and that means the other three offices will not
+ enjoy accessing it—they're likely to experience
+ significantly slower traffic and response times when
+ updating and committing code. A powerful solution is to set
+ up a system consisting of one <firstterm>master</firstterm>
+ Apache server and several <firstterm>slave</firstterm>
+ Apache servers. If you place a slave server in each office,
+ then users can check out a working copy from whichever slave
+ is closest to them. All read requests go to their local
+ slave. Write requests get automatically routed to the
+ single master server. When the commit completes, the master
+ then automatically <quote>pushes</quote> the new revision to
+ each slave server using the <command>svnsync</command>
+ replication tool.</para>
+
+ <para>This configuration creates a huge perceptual speed
+ increase for your users, because Subversion client traffic
+ is typically 80-90% read requests. And if those requests
+ are coming from a <emphasis>local</emphasis> server, it's a
+ huge win.</para>
+
+ <para>In this section, we'll walk you through a standard setup
+ of this single-master/multiple slave system. However, keep
+ in mind that your servers must be running at least Apache
+ 2.2.0 (with <command>mod_proxy</command> loaded) and
+ Subversion (<command>mod_dav_svn</command>) 1.5.</para>
<sect4 id="svn.serverconfig.httpd.extra.writethruproxy.configure">
<title>Configure the Servers</title>
- <para>Show Location block for master, *two* location blocks
- for slave. Explain why we need the second location
- block. Explain the need to keep authn and authz
- consistent across all servers.</para>
+ <para>First, configure your master server's
+ <filename>httpd.conf</filename> file in the usual way.
+ Make the repository available at a certain URI location,
+ and configure authentication and authorization however
+ you'd like. After that's done, configure each of your
+ <quote>slave</quote> servers in the exact same way, but
+ add the special <literal>SVNMasterURI</literal> directive
+ to the block:</para>
+
+ <screen>
+<Location /svn>
+ DAV svn
+ SVNPath /var/svn/repos
+ SVNMasterURI http://master.example.com/svn
+ …
+</Location>
+</screen>
+
+ <para>This new directive tells a slave server to redirect
+ all write requests to the master. (This is done
+ automatically via Apache's <command>mod_proxy</command>
+ module.) Ordinary read requests, however, are still
+ serviced by the slaves. Be sure that your master and
+ slave servers all have matching authentication and
+ authorization configurations; if they fall out of sync,
+ it can lead to big headaches.</para>
+
+ <para>Next, we need to deal with the problem of infinite
+ recursion. With the current configuration, imagine what
+ will happen when a Subversion client performs a commit to
+ the master server. After the commit completes, the server
+ uses <command>svnsync</command> to replicate the new
+ revision to each slave. But because
+ <command>svnsync</command> appears to be just another
+ Subversion client performing a commit, the slave will
+ immediately attempt to proxy the incoming write request
+ back to the master! Hilarity ensues.</para>
+
+ <para>The solution to this problem is to have the master
+ push revisions to a different
+ <literal><Location></literal> on the slaves. This
+ location is configured to <emphasis>not</emphasis> proxy
+ write requests at all, but accept normal commits from (and
+ only from) the master's IP address:</para>
+
+ <screen>
+<Location /svn-proxy-sync>
+ DAV svn
+ SVNPath /var/svn/repos
+ Order deny,allow
+ Deny from all
+ # Only let the server's IP address access this Location:
+ Allow from 10.20.30.40
+ …
+</Location>
+</screen>
</sect4>
<sect4 id="svn.serverconfig.httpd.extra.writethruproxy.replicate">
<title>Set up Replication</title>
- <para>Assumption: one master server, N slave servers.</para>
+ <para>Now that you've configured your Location blocks on
+ master and slaves, you need to configure the master to
+ replicate to the slaves. This is done the usual way,
+ using <command>svnsync</command>. If you're not familiar
+ with this tool, see <xref
+ linkend="svn.reposadmin.maint.replication"/> for
+ details.</para>
+
+ <para>First, make sure that each slave repository has a
+ <filename>pre-revprop-change</filename> hook script which
+ allows remote revision property changes. (This is
+ standard procedure for being on the receiving end of
+ <command>svnsync</command>) Then log into the master
+ server and configure each of the slave repository URIs to
+ receive data from the master repository on local
+ disk:</para>
+
+ <screen>
+$ svnsync init http://slave1.example.com/svn-proxy-sync file://var/svn/repos
+Copied properties for revision 0.
+$ svnsync init http://slave2.example.com/svn-proxy-sync file://var/svn/repos
+Copied properties for revision 0.
+$ svnsync init http://slave3.example.com/svn-proxy-sync file://var/svn/repos
+Copied properties for revision 0.
+
+# Perform the initial replication
+
+$ svnsync sync http://slave1.example.com/svn-proxy-sync
+Transmitting file data ....
+Committed revision 1.
+Copied properties for revision 1.
+Transmitting file data .......
+Committed revision 2.
+Copied properties for revision 2.
+…
- <para>Do initial svnsync from master (file:///) to each
- slave (http://). Show how.</para>
+$ svnsync sync http://slave2.example.com/svn-proxy-sync
+Transmitting file data ....
+Committed revision 1.
+Copied properties for revision 1.
+Transmitting file data .......
+Committed revision 2.
+Copied properties for revision 2.
+…
+
+$ svnsync sync http://slave3.example.com/svn-proxy-sync
+Transmitting file data ....
+Committed revision 1.
+Copied properties for revision 1.
+Transmitting file data .......
+Committed revision 2.
+Copied properties for revision 2.
+…
+</screen>
+
+ <para>After this is done, we configure the master server's
+ <filename>post-commit</filename> hook script to invoke
+ <command>svnsync</command> on each slave server:</para>
+
+ <programlisting>
+#!/bin/sh
+# Post-commit script to replicate newly-committed revision to slaves
+
+svnsync sync http://slave1.example.com/svn-proxy-sync > /dev/null 2>&1
+svnsync sync http://slave2.example.com/svn-proxy-sync > /dev/null 2>&1
+svnsync sync http://slave3.example.com/svn-proxy-sync > /dev/null 2>&1
+</programlisting>
+
+ <para>The extra bits on the end of each line aren't
+ necessary, but they're a sneaky way to allow the sync
+ commands to run in the background, so that the Subversion
+ client isn't left waiting forever for the commit to
+ finish. In addition to this
+ <filename>post-commit</filename> hook, you'll need a
+ <filename>post-revprop-change</filename> hook as well, so
+ that when a user, say, modifies a log message, the slave
+ servers get that change as well:</para>
+
+ <programlisting>
+#!/bin/sh
+# Post-revprop-change script to replicate revprop-changes to slaves
- <para>Establish post-commit and post-revprop change hook
- script to replicate each newly committed revision to each
- slave. Show scripts. Show how to background the sync
- ops, so clients aren't stuck waiting for commits to
- finish. (Also, make sure slaves are configured to accept
- propchanges.)</para>
-
- <para>Mention locks. If nothing is done, locking will work,
- but users won't be able to 'query' locks from the local
- slave servers. If you want locks to replicate to slaves,
- then the post-commit script will need to be expanded to do
- some fancy remote ssh-commands that call 'svn (un)lock' on
- slaves. Show example from jerenkrantz.</para>
+REV=${2}
+svnsync copy-revprops http://slave1.example.com/svn-proxy-sync ${REV} > /dev/null 2>&1
+svnsync copy-revprops http://slave2.example.com/svn-proxy-sync ${REV} > /dev/null 2>&1
+svnsync copy-revprops http://slave3.example.com/svn-proxy-sync ${REV} > /dev/null 2>&1
+</programlisting>
+
+ <para>The only thing we've left out here is what to do about
+ locks. Because locks are strictly enforced my the master
+ server (the only place where commits happen), we don't
+ technically need to do anything. Many teams don't use
+ Subversion's locking features at all, so it may be a
+ non-issue for you. However, if lock changes aren't
+ replicated from master to slaves, it means that clients
+ won't be able to query the status of locks
+ (e.g. <command>svn status -u</command> will show no
+ information about repository locks.) If this bothers you,
+ you can write <filename>post-lock</filename> and
+ <filename>post-unlock</filename> hook scripts which run
+ <command>svn lock</command> and <command>svn
+ unlock</command> on each slave machine, presumably through
+ a remote shell method such as SSH. That's left as an
+ exercise for the reader!</para>
</sect4>
<sect4 id="svn.serverconfig.httpd.extra.writethruproxy.caveats">
<title>Caveats</title>
- <para>Replication isn't entirely robust in the case of
- network or server crashes. If svnsync fails to happen
- after a commit, who will realize this? To make it
- bulletproof, one needs more out-of-band monitoring that
- can force an svnsync to happen... left as exercise to the
- reader.</para>
+ <para>Your master/slave replication system should now be
+ ready to use. A couple words of warning are in order,
+ however. Remember that this replication isn't entirely
+ robust in the face of computer or network crashes. For
+ example, if one of the automated
+ <command>svnsync</command> commands fails to complete for
+ some reason, the slaves will begin to fall behind. For
+ example, your remote users will see that they've committed
+ revision 100, but then when they run <command>svn
+ update</command>, their local server will tell them than
+ revision 100 doesn't yet exist! Of course, the problem
+ will be automatically fixed the next time another commit
+ happens and the subsequent <command>svnsync</command> is
+ successful—the sync will replicate all waiting
+ revisions. But still, you may want to set up some sort of
+ out-of-band monitoring to notice synchronization failures
+ and force <command>svnsync</command> to run when things go
+ wrong.</para>
+
+ <sidebar>
+ <title>Can we set up replication with
+ <command>svnserve</command>?</title>
+
+ <para>If you're using <command>svnserve</command> instead
+ of Apache as your server, you can certainly configure
+ your repository's hook scripts to invoke
+ <command>svnsync</command> as we've shown here, thereby
+ causing automatic replication from master to slaves.
+ Unfortunately, at the time of writing there is no way to
+ make slave <command>svnserve</command> servers
+ automatically proxy write requests back to the master
+ server. This means your users would only be able to
+ check out read-only working copies from the slave
+ servers. You'd have to configure your slave servers to
+ disallow write access completely. This might be useful
+ for creating read-only <quote>mirrors</quote> of popular
+ open-source projects, but it's not a transparent
+ proxying system.</para> </sidebar>
</sect4>
@@ -2611,7 +2811,7 @@
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<sect3 id="svn.serverconfig.httpd.extra.other">
- <title>Other Features</title>
+ <title>Other Apache Features</title>
<para>Several of the features already provided by Apache in
its role as a robust Web server can be leveraged for
More information about the svnbook-dev
mailing list