2 September 2013 Roberto Puzzanghera0 comments
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon.
It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.
Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time.
Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file's data does not need to be updated.
I will show shortly how to:
Before we start, I'll call "local" the computer where the files have to be copied and "remote" the computer where those files are stored and where you have to listen for ssh connections.
To secure our data, we'll use rsync via a remote ssh connection, so there's no need to start rsync as a daemon, but sshd must be configured to accept connections without password and rsa-key authentication must be enabled in your /etc/ssh/sshd_config file:
AllowUsers ssh-user RSAAuthentication yes
Here "ssh-user" is the only user who is allowed to connect via ssh. If the backup copy has to be done from a mirror web server, it is not secure to store anywhere an ssh-rsa key which enables a connection as root; this is the reason why we are going to do the connection with an unpriviledged user "ssh-user".
So "ssh-user" will be used at the ssh level and should not be confused with "rsync-user", which will be used to log-in to the rsync "module", site1 in the following example.
Log-in as "ssh-user" and create the config file ~/rsync.conf. Be aware that the rsync process started by the ssh shell, if run by a user other than root, expects the config file in the home directory of the ssh user.
# common stuff motd file = /etc/rsyncd_motd # the following in case you want to test rsync as daemon log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsyncd.lock [site1] # this is the path of the files to backup path = /home/ssh-user/path/where/site1/files/live comment = site1 files uid = ssh-user gid = apache read only = yes list = yes auth users = rsync-user secrets file = /home/ssh-user/rsyncd.scrt # we don't have super user access use chroot = false [site2] [....site2 stuff....]
uid and gid are the userID and the groupID under which file transfers will take place.
Before the transfer will start you have to authenticate rsync with "auth user". Create the secret file ~/rsync.scrt which holds the user:password couples:
Remove the 'r' flag to other users:
chmod o-r ~/rsync.scrt
Since we want to backup our files by means of a script and a cronjob, it's important that the remote ssh connection will not prompt for any password. We can do this exchanging a ssh-key between client and server.
Create the private and public key:
root@localhost:~# ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa_remoteHost): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa_remoteHost. Your public key has been saved in /root/.ssh/id_rsa_remoteHost.pub. The key fingerprint is: a0:53:33:c5:d1:ea:4c:e2:a1:98:d9:ba:b0:e8:5f:90 root@localhost The key's randomart image is: +--[ RSA 2048]----+ | o++o | | o. . | | . .. | | .oo. | | E.O .S | | * * | |. . o . | |.o. . . | |+.oo | +-----------------+
Now copy the public key id_rsa_remoteHost.pub to the remote server. ssh-copy is a program which will do the connection and will append the key to ~/.ssh/authorized_keys of the root user, but in our case we have to append to the file
because the connection will be done by "ssh-user" and not by root, as explained earlier.
Now test that the connection is allowed with no password:
root@localhost:~# ssh -p 12345 -l ssh-user -i /root/.ssh/id_rsa_remoteHost Last login: Mon Sep 2 16:04:57 2013 from localhost Linux 18.104.22.168-vs22.214.171.124.29.2-smp. ssh-user@remotehost:~#
Now we are ready to create our backup shell-script as /usr/local/bin/rsync_backup.sh:
#!/bin/sh /usr/bin/rsync \ -avz --exclude "*~" --delete-after \ -e "ssh -p 12345 -l ssh-user -i /root/.ssh/id_rsa_remoteHost" \ --password-file /root/remoteHost_rsync_pwd \ rsync-user1@::site1 \ /local/destination/path
Remember to give the flag +x to that file:
chmod +x /usr/local/bin/rsync_backup.sh
The password file /root/remoteHost_rsync_pwd holds the password of the rsync connection; in this way our shell-script will not receive a password prompt when it connects. It should be stored in a safe place and priviledges must be given only to the root user. It will contain just the password string.
Maybe the line
-avz --exclude "*~" \
deserves some description, but you are invited to refer to the man page for more details.
You can have a quick connection to the remote Host if you setup a ~/.ssh/config file as follow
Host HostName User root Port 12345 IdentityFile ~/.ssh/id_rsa_remoteHost
and connecting as
> ssh Enter passphrase for key '/home//.ssh/id_rsa_remoteHost': Last login: Mon Sep 2 16:04:57 2013 from localhost Linux 126.96.36.199-vs188.8.131.52.29.2-smp. ssh-user@remotehost:~#
At this point it is convenient to disable root remote access setting
/etc/ssh/sshd_config as follow:
PermitRootLogin without-password AllowUsers root PubkeyAuthentication yes
apache clamav dkim dovecot ezmlm fail2ban hacks lamp linux linux-vserver mariadb mediawiki mozilla mysql owncloud patches php proftpd qmail qmailadmin rbl roundcube rsync sieve simscan slackware spamassassin ssh surbl tcprules tex ucspi-tcp vpopmail vqadmin