SME Server RSYNC Instructions
Using rsync to obtain the e-smith CD ROM image
We highly recommend using rsync as the method of choice for obtaining the e-smith CD ROM image. We have found it to be the most efficient and least bandwidth-intensive method for downloading any large file(s). In a nutshell, rsync only downloads the differences between a file at the source location and a file of the same name at the destination. When dealing with a 300+ MB file like the e-smith CD ROM image, this can result in download savings upwards of 80%.
This document provides links to information about rsync, and instructions on how to use it to obtain the e-smith server and gateway software.
Why should you use rsync?
In the broadest terms, anyone using with an Internet-connected Linux machine should seriously consider using rsync. Even if bandwidth savings don't mean anything to you individually, the bandwidth savings that can be achieved using rsync benefit everyone connecting to the download resource.
What is rsync?
First, rsync is not by nature Windows software. While there are documents online that explain how to build rsync on Windows 95 and Windows NT, only advanced users who are comfortable compiling their own software should even try.
rsync does run on Linux and many Linux-like operating systems *grin*. You can find binary versions for 10 or so different operating systems and Linux distributions at http://rsync.samba.org/ftp/rsync/binaries/.
From the rsync home page:
"rsync is an open source utility that provides fast incremental file transfer. rsync is freely available under the GNU General Public License."
The following is taken from the rsync man page:
rsync is a program that behaves in much the same way that rcp does, but has many more options and uses the rsync remote-update protocol to greatly speedup file transfers when the destination file already exists.
The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network link, using an efficient checksum-search algorithm described in the technical report that accompanies this package.
Some of the additional features of rsync are:
* support for copying links, devices, owners, groups and permissions
* exclude and exclude-from options similar to GNU tar
* a CVS exclude mode for ignoring the same files that CVS would ignore
* can use any transparent remote shell, including rsh or ssh
* does not require root privileges
* pipelining of file transfers to minimize latency costs
* support for anonymous or authenticated rsync servers (ideal for mirroring)
How do I learn about rsync?
The first place to start learning about rsync is of course the man(ual) page. Simply enter man rsync at your linux command prompt.
You can look at some of the examples at http://rsync.samba.org/rsync/examples.html.
There's a very good tutorial on how to use rsync located at http://www.eunuchs.org/linux/rsync/. NOTE: This site seems to be offline right now.
Finally, for the more technically inclined, the rsync resource pages contain a number of other useful links.
How do I use rsync to obtain the e-smith CD ROM image?
Follow these steps to download the e-smith CD ROM image:
1. If you have an older copy of the ISO image (for example a beta or release candidate) you can use it to prime the destination file. Copy the old CD ROM image to a new name. For example, if you have an image of smeserver 5.6 beta 2, you would type the following:
cp smeserver-5.6b2unsupported.iso smeserver-5.6unsupported.iso
2. Next, select the rsync mirror nearest you. See the download page for a listing. In the following example, we'll assume you've chosen the North American mirror site. Be sure to check for a mirror closer to you, though.
rsync -avv --stats \
rsync://ftp.ibiblio.org/Linux/distributions/e-smith/current/iso/smeserver-5.6unsupported.iso .
Note that the backslash character at the end of the first line indicates that the second line is actually a continuation of the first.Note also that there is a dot ('.') character at the end of the second line.
The output from this command should look something like this:
Number of files: 1
Number of files transferred: 1
Total file size: 329203712 bytes
Total transferred file size: 329203712 bytes
Literal data: 50102272 bytes
Matched data: 279101440 bytes
File list size: 52
Total bytes written: 117913
Total bytes read: 50176944
wrote 117913 bytes read 50176944 bytes 179945.82 bytes/sec
total size is 329203712 speedup is 6.55
In the example above, the total bytes written is tiny compared to the actual size of the ISO image. The speedup of 6.55 (i.e. rsync was 6.55 times faster than a straight download) means that the bandwidth and time taken was about 15% of what it might have been!
Important Note: Be prepared to have space temporarily for two images in your local directory during the rsync.
Additional command-line options to consider
-z (compress):
While we have had some intermittent problems with the -z (compress) option with rsync and the large iso images, we do recommend trying it. Be sure to check the file size and the MD5Sum before burning the ISO to CD though. A corrupt CD image can potentially do damage to your server.
-P --partial:
The --partial option instructs rsync to leave any partial file(s) that might have been created during an interrupted download. This is especially useful for those with intermittent or unreliable connections to the Internet, because it means that they can resume their download from the place they left off. See the rsync man pages for more details on how to apply this option.
Back to Downloads