From 48ab4dd1e7b4dad30d04a593b709940d465eee61 Mon Sep 17 00:00:00 2001 From: "Hahn Axel (hahn)" <axel.hahn@iml.unibe.ch> Date: Fri, 30 Aug 2019 16:50:12 +0200 Subject: [PATCH] add .gitignore and first lines of docs --- .gitignore | 4 +- jobs/README.md | 113 +++++++++++++++++++++++++++++++++++++++++++++++++ keys/.gitkeep | 0 logs/.gitkeep | 0 4 files changed, 116 insertions(+), 1 deletion(-) create mode 100644 jobs/README.md create mode 100644 keys/.gitkeep create mode 100644 logs/.gitkeep diff --git a/.gitignore b/.gitignore index de63fad..fe08cd8 100644 --- a/.gitignore +++ b/.gitignore @@ -2,5 +2,7 @@ # ignored configs # ---------------------------------------------------------------------- jobs/*.job -logs/* +logs/full-* +logs/inc-* +logs/transfer-* *lastlog \ No newline at end of file diff --git a/jobs/README.md b/jobs/README.md new file mode 100644 index 0000000..3cfa11b --- /dev/null +++ b/jobs/README.md @@ -0,0 +1,113 @@ +# IML BACKUP # + +Backup scripts using duplicity. +Runs on Linux. + +Free software. GNU GPL 3.0. + +Source: https://git-repo.iml.unibe.ch/iml-open-source/iml-backup/ +Duplicity: http://duplicity.nongnu.org/ + +## Why ## + +We don't want to configure a backup set on a "central backup server" for each new node. Each new node pushed its own backup data to a backup target. + +We want to push data from a private network to target; a central backup server would ot reach some clients. + +A set of database backup scripts detects exsiting locally running database servers and puts a compressed dump file per database scheme to a local backup directory. + +Then a transfer script uses duplicity to encrypt and transfer local backups and other local folders to a backup target. + +## Features ## + +Supported Databases for backup and restore + +* Mysql/ Mariadb (mysql_dump) +* postgres (pg_dump) +* sqlite (by naming files with full path in a config) + +Limited support: + +* couchdb (using a config with naming convention) +* ldap (without restore so far) + +Duplicity allows + +* Incremental and full backups +* encrypted backups using GPG +* set size of backup volumes +* delete old backups by a given time limit +* several backup targets (we currently use scp:// rsync:// and file://) + +## Installation ## + +- Uncompress / clone the client to a local directory +- go to jobs directory to copy the *.job.dist files to *.job +- configure *.job files +- manual test run +- create a cronjob + +### Uncompress client ### + +To put all files into a directory i.e. + + /opt/imlbackup/client + +then use the **root** user and follow these steps: + + # Create the directory level above + mdir -p /opt/imlbackup/ + + # download + cd /opt/imlbackup/ + wget https://git-repo.iml.unibe.ch/iml-open-source/iml-backup/-/archive/master/iml-backup-master.tar.gz + + # extract + tar -xzf iml-backup-master.tar.gz + mv iml-backup-master client + + # remove downloaded file + rm -f iml-backup-master.tar.gz + + # to set pwd to /opt/imlbackup/client: + cd client + +### database backup: set local backup target ### + +Create a jobs/dirs.jon (copy the deliverers *.dist file) + + cd jobs + cp dirs.job.dist dirs.job + cd .. + +There are 2 defaults: + + dir-localdumps = /var/iml-backup + keep-days = 7 + +**dir-localdumps** +{string} +The target directory for local dumps. It is used by + +* the database dump scripts +* the transfer script to store the client backups +* the restore script + +Below that one a directory for the service will be generated; inside that one the database dumbs with scheme name and timestamp, i.e. + + /var/iml-backup/mysql/mydatabase__20190827-2300.sql.gz + +**keep-days** +{integer} +The number of days how long to keep dumps locally. + +Remark: +To make a database restore its dump must be located at this directory. To restore an older database you need to restore the dump from duplicity first. + +If you have local Mysql daemon or Pgsql you can test it by starting + + # dump all databases + ./localdump.sh + + # show written files + find /var/iml-backup diff --git a/keys/.gitkeep b/keys/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/logs/.gitkeep b/logs/.gitkeep new file mode 100644 index 0000000..e69de29 -- GitLab