Harvest data using rsync
Slow going today – its raining in London and I’ve got lots to do with the US side of the business which I’m not too fussed about today so I’ve decided to harvest all live data from a server I’m trying to decommission. First stage in this is to get all the data off the server locally to my machine which I can then backup or offload to another server or maybe even push to S3 depending on the data.
The script is using rsync and connects via ssh, the important points of the script is using OPTS to exclude files that we don’t want copy down in this case it’s zips and temp files. We are also using nohup just in case the SSH shell crashes so it still runs in the background.
The end of the script has a simple email function to email the logs when complete or when it errors
OPTS="-vaz --exclude=/zip/exports --exclude=/temp/_exports"
cat /dev/null > /tmp/rsync-errors.txt
nohup $RSYNC $OPTS $SOURCE $TARGET >> /var/log/rsync-raw-out.txt 2>> /tmp/rsync-errors.txt
if [ -s '/tmp/rsync-errors.txt' ]; then
mail -s "Errors on $HOSTNAME" < /tmp/rsync-errors.txt $MAIL_ADDR
echo "Harvest successful, no errors reported." | mail -s "Harvest Report for $HOSTNAME" $MAIL_ADDR