update: safety first!

when it comes to important files: safety comes first

thus recommend to the user that wants to go pro the following backup system:

  • have two complete backups at two different places:
    • backupA: at the company, USB 3.0 (! THE MORE DATA IS BACKED UP THE MORE SPEED IS NEEDED (or restore might take DAYS!)) connected to the server, doing daily incremental backups
    • backupB: being a fire-proof double-metal casing (EMP proof) vault at a different place (home?)
  • change those backups every day if possible otherwise every week
    • if ransomeware destroys backupA then in the worst case scenario, one day or one week of work is gone
    • remember: whatever is physically connected to the server, can be encrypted by ransomeware
  • have the backup strategy tested once a year
    • where the backup is restored completely on a backup-server, to test if all data is there and how long the process takes (USB 2.0 is definitely a massive bottleneck)

about:

this script backs up your entire system over internet to a ssh reachable (rsync needs to be installed) fileserver destination over internet.

it will not do incremental backups! (just one copy of what you exactly have)

it will not delete files on backup destination deleted from origin (so you can recover accidentally deleted files, but have to tidy up on backup destination manually)

imho if you backup a virtualization server – regularly tidy up your harddisk space or internet backups will take very long!

to consolidate disk space of virtualbox vms, simply: powerdown and clone them! (attach a version number or something, also i would keep the mac addresses)

for example if many snapshots were done, this webserver needs 115GBytes of space to 59Gbytes (almost 50% disk space saved!)

preparation:

the script is running as root, so there are no file permission problems.

# option: -x : stay on one file system = it will not backup mounted drives!

test if public key authentication works (here with user admin@123.123.123.123) my manually logging in (needs to work password-less)

private key used: /home/backup/.ssh/id_rsa

on source (server to backup) create script and log folder and script:

mkdir /scripts

mkdir /scripts/log

vim /scripts/backup.sh

insert this

chmod +x /scripts/backup.sh

run it

/scripts/backup.sh

#!/bin/bash
SERVICE="rsync"

# test if rsync is still doing a backup
if pgrep -x "$SERVICE" >/dev/null
then
# then do not start another backup
	echo "$(date '+DATE: %Y-%m-%d TIME: %H:%M:%S') $SERVICE backup is in progress, only one backup process shall operate at the same time, log goes to /scripts/log/date_backup_still_in_progress.log"

	echo "------- $(date '+DATE: %Y-%m-%d TIME: %H:%M:%S') backup still in progress log:" >> /scripts/log/$(date '+%Y-%m-%d')_backup_still_in_progress.log;
else
	echo "starting new $SERVICE remote backup, log goes to /scripts/log/date_backup_started.log"

	echo "------- backup started at date $(date '+DATE: %Y-%m-%d TIME: %H:%M:%S') log:" > /scripts/log/$(date '+%Y-%m-%d')_backup_started.log;

# limiting bandwidth usage to 2048KByte/s = 2MByte/s

	rsync -avxHAX -r --update --partial --checksum --inplace --progress --bwlimit=2048 /* -e 'ssh -v -i /home/backup/.ssh/id_rsa' admin@123.123.123.123:/share/MD0_DATA/path/to/BACKUP/ --exclude={*swapfile*,/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/home/*/.gvfs} >> /scripts/log/$(date '+%Y-%m-%d')_backup_started.log;
fi

# options explained:
# -a, archive mode, all files, with permissions, equals -rlptgoD (no -H,-A,-X)
# -v  : verbose, mention files
# -x  : stay on one file system
# -H  : preserve hard links (not included with -a)
# -A  : preserve ACLs/permissions (not included with -a)
# -X  : preserve extended attributes (not included with -a)

# if mail is installed and working, you can mail log about backup
# cat /scripts/log/$(date '+%Y-%m-%d')_backup.log | mail -s "domain.com - online backup report - rsync.log" admin@domain.com;

check the progress:

you can check the progress/what the script is doing with tail -f (follow changes)


tail -f /scripts/log/2019-06-30_backup_started.log 
------- backup started at date DATE: 2019-06-30 TIME: 13:52:40 log:
sending incremental file list
root/
root/.viminfo
          9,105 100%    8.02MB/s    0:00:00 (xfr#1, ir-chk=1206/6379)
scripts/
scripts/backup.sh
          1,473 100%    1.40MB/s    0:00:00 (xfr#2, ir-chk=1144/6379)

liked this article?

  • only together we can create a truly free world
  • plz support dwaves to keep it up & running!
  • (yes the info on the internet is (mostly) free but beer is still not free (still have to work on that))
  • really really hate advertisement
  • contribute: whenever a solution was found, blog about it for others to find!
  • talk about, recommend & link to this blog and articles
  • thanks to all who contribute!
admin