A Simple Website Backup

I  try to keep things simple and inexpensive. Instead of paying for backups I roll my own. Now I do pay for network bandwidth so now I have to make sure I keep the network usage to a minimum. My simple website backup solution is as follows

  1. Use rsync to backup the important files from my hosted web site to my home computer
  2. backup the  remote host backup that now resides on a local disk as a mirror image of selected directories to another disk in an incremental fashion.

Assumptions

  1. In the case of a complete disaster, we will have to rebuild the server. Using digitalocean, this takes 55 seconds.
  2. It is possible to use the digitalocean options to install wordpress and other files. But we wont use that assumption here.
  3. We will assume that you have documented clearly how to install all the required applications.
  4. Any config files changes to the basic installation will be in your backups.
  5. Your backups will contain all the data necessary to restore your site. This is the stuff you backup

rsync is a program that only transfers what has changed. It has a lot of options. I recommend reading the man page.
I created the following bash script which runs as a normal user and is run through a scheduler such as cron. It will use sudo on the remote system to backup files that are not accessible to a non-privileged user. You don’t want it to run as root becuase root is not allowed to login via ssh.
It simply backs up all the important directories to one location. It essentially makes a mirror.
I then use the backup script backup-manager.sh on the home system to make full and incremental backups.

Requirements

  1. rsync on remote and local systems
  2. sudo on the remote system.
  3. cron facility on the local system
  4. backup-manager on the local system
  5. ssh keys
  6. Linux or Unix operating system

backup script that runs on the home computer:

You will have to modify the script depending upon what you want backed up and what permissions you are allowed.

#!/bin/bash 
# "@(#)$Header: remote_backup.sh V1.0 - By: msimoni 
###############################################################################
# Name : Michael Simoni
# date : Tue Oct 21 10:25:17 EDT 2003
# Purpose: to backup a data to a second disk
################################################################################
# CVS Information
# Source : $
# $Author: simoni $
# Date of last commit: $Date: 2015/01/03 20:16:28 $
# ID: $Id: remote_backup.sh,v 1.4 2015/01/03 20:16:28 msimoni Exp $
# Revision: $Revision: 1.4 $
###############################################################################
# Notes: 
# rsync -avz -e "ssh -p $REMOTE_PORT -l i$REMOTE_USER -i $RSA_RSYNC_KEY" $REMOTE_USER@$REMOTE_HOST:$PATH_TO_BACKUP $LOCAL_BACKUP_REPO
###############################################################################
###############################################################################
# AUTHOR: Michael Simoni (ms), mdsimoni@zebulak.com
# COMPANY: Zebulak Enterprises LLC
# VERSION: $Revision: 1.4 $
# CREATED: Tue Oct 21 10:25:17 EDT 2003
# REVISION: $Revision: 1.4 $
# COPYRIGHT: Copyright (c) 2013, Michael Simoni
#===============================================================================
# @see The GNU Public License (GPL): http://www.opensource.org/licenses/gpl-license.php
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
###############################################################################
# usage
###############################################################################
usage()
{
cat <<-EOF
 $PROGNAME: -p port -u username -r remotehost -k key_file
 f [logfile] changesthe default LOGFILE: default: $LOGFILE
 u [username] the username that you will login to the remote host
 r [remote host] the remote host to loginto
 p [port] the port that ssh is using on the remote host
 k [key_file] the private key file that is used for ssh to loginto
 m [mail-recipient] mail to a local user
 d) set debug. outputs to screen instead of the log file
 h) this message
EOF
}

################################################################################
# my data backup
# --delete-excluded \
# --include-from=$INCLUDE_FILE \
# --delete-after \ 
# --ignore-errors \
# ${REMOTE_HOST}:/var/log \
################################################################################
my_data_backup()
{
if [ $DEBUG -eq 1 ]
then
 set -x
fi
#time $RSYNC -4 \
# -avz \
# --delete \
# --delete-excluded \
# --exclude-from=$EXCLUDE_FILE \
# --rsync-path="sudo rsync" \
# -e "ssh -p $REMOTE_PORT -l $REMOTE_USER -i $REMOTE_KEY" \
# ${REMOTE_HOST}:/root \
# ${REMOTE_HOST}:/home/ \
# ${REMOTE_HOST}:/var/www \
# ${REMOTE_HOST}:/etc \
# $BACKUP_DIR/

# run the rrsync and the sudo command on the remote end, from the authorized_keys file
time $RSYNC -4 \
 -avz \
 --delete \
 --delete-excluded \
 --exclude-from=$EXCLUDE_FILE \
 --rsync-path="/usr/bin/sudo /usr/bin/rsync" \
 -e "/usr/bin/ssh -o IdentitiesOnly=yes -p $REMOTE_PORT -l $REMOTE_USER -i $REMOTE_KEY" \
 ${REMOTE_HOST}:/root \
 ${REMOTE_HOST}:/home \
 ${REMOTE_HOST}:/var/spool/cron \
 ${REMOTE_HOST}:/var/www \
 ${REMOTE_HOST}:/etc \
 $BACKUP_DIR/

#gzip -9fv $BACKUP_DIR/log/httpd/*log
}
################################################################################
# create a file of exclude list
################################################################################
my_exclude_list()
{
cat >$EXCLUDE_FILE <<-EOF
- /etc/selinux/
- /root/.cpan
EOF
}
################################################################################
# create a file of include patterns
################################################################################
my_include_list()
{
cat >$INCLUDE_FILE <<-EOF
+ /var/www 
+ /etc
- /var/www/cache
EOF
}
################################################################################
# Main
################################################################################
PROGNAME=$0
NODE=`uname -n`
RSYNC=/usr/bin/rsync
LOGFILE=/tmp/remote_backup.log
THEDATE=`/bin/date +%F-%R:%S`
EXCLUDE_FILE=/home/userid/dev/backup/rsync_exclude_file
INCLUDE_FILE=/home/userid/dev/backup/rsync_include_file
REMOTE_HOST=""
MAILTO=""
#
REMOTE_USER=""
REMOTE_PORT=""
#REMOTE_KEY=/home/msimoni/.ssh/id_rsa_rsync
REMOTE_KEY=""
BACKUP_DIR=/u02/backups/srv1
DEBUG=0
#
if [ $# -eq 0 ]
then
 usage
 exit
fi

while getopts h?df:k:m:p:r:u: OPT
do
 case $OPT in
 f) LOGFILE=$OPTARG ;;
 u) REMOTE_USER=$OPTARG ;;
 r) REMOTE_HOST=$OPTARG ;;
 p) REMOTE_PORT=$OPTARG ;;
 k) REMOTE_KEY=$OPTARG ;;
 m) MAILTO=$OPTARG ;;
 d) DEBUG=1 ;;
 h) usage ; exit 1 ;;
 ?) usage ; exit 1 ;;
 *) usage ; exit 1 ;;
 esac
done
shift $(($OPTIND - 1))
#echo "Remaining arguments are:\n" "$*"

if [ $DEBUG -eq 1 ]
then
 echo "debug set: output going to screen"
else
 > $LOGFILE
 exec 1>>$LOGFILE
 exec 2>>$LOGFILE
fi
echo "Starting a backup at `date`"
if [ -z "$REMOTE_USER" -o -z "$REMOTE_HOST" -o -z "$REMOTE_PORT" -o -z "$REMOTE_KEY" ]
then
 echo "REMOTE_USER: $REMOTE_USER"
 echo "REMOTE_HOST: $REMOTE_HOST"
 echo "REMOTE_PORT: $REMOTE_PORT"
 echo "REMOTE_KEY : $REMOTE_KEY"
 echo "ERROR: Need user or host or port or ssh key set. Exiting."
 exit
fi

echo "syncing / ...."
my_exclude_list
my_include_list
my_data_backup
echo "Ending backup at `date`"

if [ -n "$MAILTO" -a $DEBUG -eq 0 ]
then
 # send a mail to a local user
 cat $LOGFILE | /usr/bin/mailx -s "$NODE wordpress backup" $MAILTO
fi
exit
	

 backup-manager setup

Once you get that set up and working the next step is to install backup-manager on your home system. You should install a secondary disk for system backups.

yum install backup-manager.noarch

Once you install the backup manager and the seconday disk start backing up your system files and of cours your directory where you backed up your hosted website.

you will want to modify the following variables:

BM_REPOSITORY_ROOT
# this is my setting for the archive method because I have mysql 
# running on my home computer
BM_ARCHIVE_METHOD="tarball-incremental mysql"
BM_TARBALL_DIRECTORIES
BM_TARBALL_BLACKLIST

 crontab setup

You will have to modify the parameters for your environment.

################################################################################
 # * * * * * command to be executed
 # - - - - -
 # | | | | |
 # | | | | +----- day of week (0 - 6) (Sunday=0)
 # | | | +------- month (1 - 12)
 # | | +--------- day of month (1 - 31)
 # | +----------- hour (0 - 23)
 # +------------- min (0 - 59)
 ################################################################################
 # remote backups - wordpress
 ################################################################################
 00 23 * * * /home/backup/remote_backup.sh -u theuser -p 22 -r hostname -k /pathto/.ssh/key -m localuser

sudo setup

Use visudo to edit the sudoers file on the remote host

# Disable "ssh hostname sudo <cmd>", because it will show the password in clear.
 # You have to run "ssh -t hostname sudo <cmd>".
 #
 Defaults requiretty
 Defaults:youruserID !requiretty

# Add the following:

userID ALL=SETENV:NOPASSWD: /home/userid/bin/rrsync -ro /

Set up your ssh-keys

use ssh-keygen
You will have to use a key on that does not have a passphrase.
It may be possible to further restrict what this key can do on the hosted site but I have not played enough with that.

There is a script called rrsync that is distributed with rsync. It is used to limit uploads and downloads of users to specific directories and to directories that the users own. For my purposes i Needed to backup files across the entire file system and owned by anyone including root. So i need to use the sudo command and the settings above. for reference the rrsync command is setup in the .ssh/authorized_keys file as follows:

command="sudo -E $HOME/bin/rrsync -ro /",no-agent-forwarding,no-port-forwarding,
no-pty,no-user-rc,no-X11-forwarding ssh-rsa 
ALL-THE-KEY-LETTER-GOBBELDY-GOOK-STUFF-HERE username@hostname Automated remote backup

So if you need to back up across the entire file systems the key entry shuold look like similar to this:

no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa 
ALL-THE-KEY-LETTER-GOBBELDY-GOOK-STUFF-HERE username@hostname Automated remote backup

Remember to test through cron not through the command line.

Troubleshoooting

When i tried to use the rrsync file I was getting some errors like this:

rsync: opendir “/var/spool/cron” failed: Permission denied (13)
This was fixed once I modified the backup script by replacing the following line

–rsync-path=”rsync” \
with
–rsync-path=”/usr/bin/sudo /usr/bin/rsync” \

however the forgoing only worked when you invoked it from the command line. When it was executed through cron this error appeared:

/home/msimoni/bin/rrsync: SSH_ORIGINAL_COMMAND=’sudo rsync –server –sender -vlogDtprze.iLs . /root /home /etc ‘ is not rsync

This is due to this line in the rrsync code:

die "$0: SSH_ORIGINAL_COMMAND='$command' is not rsync\n" unless $command =~ s/^rsync\s+//;

The other error was:

sudo: no tty present and no askpass program specified
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.0]

This was fixed once I updated my sudo rules with /usr/bin/rsync. See above.


Leave a Reply

Your email address will not be published. Required fields are marked *