10_Features.md

• Automatic backup of databases: A set of database backup scripts detects exsiting locally running database services and puts a compressed dump file per database scheme to a local backup directory.
• local encryption of all data before transferring to backup storage
• no agent, no sattelite, no central backup server. We deploy the software and a config set to a server and the backup will run itself
• you can add custom scripts in many hooks to react if actions failed or were ok
• supported restore actions:
  • restore files files from backup storage
  • restic on linux: mount restic repository with fuse - then you can browse through snapshots and data like in a normal filesystem
  • restore a single database with some comfort
• a check script shows the local Backupstatus

How does it work?

For backup it makes

• the dump of all found local databases and then
• transfer all files with a backup tool:
  • locally encrypted files will be transferred to a mounted dir, ssh, rsync, https, (the available backaneds depend on the used backup tool) ... to a backup target system
  • on the backup target each system has its own subdirectory or repository to store its data

graph LR
    Start((Start)) --> | Cronjob<br>in the night | backup.sh(backup.sh:<br><br>init config)

    backup.sh --> localdump.sh(localdump.sh:<br><br>Dump<br>local<br>databases)
    localdump.sh --> transfer.sh(transfer.sh:<br><br>Secure<br>file backup<br>using Restic<br>or Duplicity)
    transfer.sh --> End((End))

Dump local databases

Before starting the backup of local files to a backup target there is a step to dump a local database. If one of the supported database types is detected it will be dumped.

We try to implement the approach "zero config" if possible.

Supported local databases for backup and restore:

• Mysql/ Mariadb (requires mysql_dump)
• postgres (pg_dump)
• sqlite (by naming files with full path in a config)

Limited support:

• couchdb (using a config with naming convention)
• ldap (without restore so far)

graph LR

    %% --- define nodes ----------

    localdump.sh(localdump.sh)
    loopdbs((Loop over<br>dabase types))
    db1[(mysql)] 
    db2[(pgsql)] 
    db3[(...)] 

    db1A[db A]
    db1B[db B]
    db1C[db C]

    subgraph /var/iml-backup/mysql/
      dump1A[dump A]
      dump1B[dump B]
      dump1C[dump C]
    end

    %% --- connect nodes ----------

    localdump.sh --> loopdbs
    loopdbs --> db1
    loopdbs --> db2
    loopdbs --> db3

    db1 --> db1A
    db1 --> db1B
    db1 --> db1C

    db1A --> dump1A
    db1B --> dump1B
    db1C --> dump1C

File backup

It is a classic backup a set of directories. Includes and excludes can be defined (but I prefer to backup all files).

The backup target for local database backups is included automatically.

Additionally there is a flag for a local Samba: if enabled all found shares will treated as directory to backup.

graph LR

    %% --- define nodes ----------

    transfer.sh(transfer.sh)
    loopdirs((Loop over<br>set of dirs))

    dir1(/etc)
    dir2(/home)
    dir3(/var/www)
    dirA(/var/iml-backup/mysql)
    dirB(...)

    backup(Backup<br>Restic or<br>Duplicity)
    backuptarget(Remote Backup target<br>in the cloud<br>or on another<br>physical location)
    verify(Verify<br>written<br>data)

    %% --- connect nodes ----------

    transfer.sh --> loopdirs

    loopdirs --> dir1
    loopdirs --> dir2
    loopdirs --> dir3
    loopdirs --> dirA
    loopdirs --> dirB

    dir1 --> backup
    dir2 --> backup
    dir3 --> backup
    dirA --> backup
    dirB --> backup

    backup --> |transfers<br>encrypted<br>files| backuptarget
    backuptarget --> verify

Hooks

You have several entry points to execute your custom scripts before, after and during th backup process. See Hooks

Backup tools

Restic

• creates one initial full backup - and then never again.
• encrypts data
• deduplicates files
• delete backups by rules to keep a count of hourly, daily, weekly, mothly, yearly backups
• several backup targets (currently supported protocols: sftp:// https:// and file://)
• Single binary (written in Go)

Duplicity

• Incremental and full backups
• encrypted backups using GPG
• set size of backup volumes
• delete old backups by a given time limit
• several backup targets (currently supported protocols: scp:// rsync:// and file://)
• Backup script (written in Python) with dependencies and possibly differend versions on different OS

Limit simoultanous backups

As an optional feature you can control the count simultanous written backups. This requires additional effort next to the client installation.