The file transfer describes how local directories will be backed up

* choose backup tool; incremental or full backup
* where to write data (backup target)
* set password for local encryption
* more options
* how to cleanup old backup data

Details to the the given config entries you find in the description for
[transfer.job](50_File_transfer.job.md).

## Backup tool ##

You can decide between Restic (Default) and Duplicity.

`bin = ...`

```text
bin = restic
type = auto
```

If you use Duplicity for servers it is suggested to run (daily) incremental
backups and full backups on a specific day.

```text
bin = duplicity
type = inc
full = WDM:3rd Fri
start-time-full = 23:0.
```

If you use Duplicity on a desktop client you maybe want to run incremental backups
and a full backup if the last one is older than a wanted time.

```text
bin = duplicity
type = auto
auto = 1M
```

## Backup target ##

The definition of a backup target depends on the used backup tool (restic|duplicity).
You need a bit knowledge about these tools.

We have tested the following targets

* local directory/ mounted filesystem i.e. Samba share
  * Restic: `/mnt/mounted_directory`
  * Duplicity: `file:///mnt/mounted_directory`
* ssh target (scp or rsync)
  * Restic: `sftp:imlbackup@backup-target.example.com:/mnt/backupdata`
  * Duplicity: `rsync://imlbackup@backup-target.example.com//mnt/backupdata`
  * Duplicity: `scp://imlbackup@backup-target.example.com//mnt/backupdata`
* https
  * Restic: `rest:https://user:pass@backup-target.example.com:8000/user/`

Set your value in `storage = ...`

Example for Restic with SFTP transfer

`storage = sftp:imlbackup@backup-target.example.com:/mnt/backupdata`

## Local encryption ##

Restic an Duplicity encrypt local data with a password before transferring them.

Deploy a host specific password with an orchestration tool (Ansible, Puppet, ...)
or on a manual installation keep a copy of it on a safe place. Without the password
you cannot decrypt backup data after a filesystem crash.

`passphrase = EnterYourSecretHere`

## More options ##

Automatic backup of samba shares - it works only if samba shares were detected.
If no samba config is available it has no effect and shows no error.

`sambashares = 1`

When restoring data ... this is the path for restored files:

`restore-path = /restore`

### Duplicity settings ###

When writing backup data duplicity creates the target file and a index.
By default the index is written to ~/.cache/duplicity/ (and to the backup target).
If your /root partition has a limited size and/ or you have a lot of files
(i.e. on a file server) then you can put the index files somewhere else:

`duplicity_cachedir = [full path]`

Duplicity with rsync needed an additional ssh-backend *pexpect* on Debian.
It can be set with enabling:

`# duplicity_ssh-backend = pexpect`

On the backup target you can define the size of the chunks to write.
This would allow to write the backup data to an additional medium with
limited size (i.e. CD or DVD).

Set a size in MB

`duplicity_volsize = 350`

### Restic settings ###

When writing backup data restic creates a local index.
By default the index is written to ~/.cache/restic/.
If your /root partition has a limited size and/ or you have a lot of files
(i.e. on a file server) then you can put the index files somewhere else:

`restic_cachedir = [full path]`

Restic by default stores file indexes locally. To save space in `restic_cachedir`
you can disable the file index but the backup will run much slower.

`restic_nochache = false`

The default verbose of IML backup for restic is 0. 
If you want to see unchanged/ new/ deleted files set it to "2". But This generates
larger logs in [installdir]/logs/.

`restic_verbose = 0`

A file transfer can be tagged. Each restic backup gets a defined tag

`restic_tag = imlbackup`

For restore you can copy files to the restore location. With restic
you can mount the backup volume into the local filesystem using fuse.
Then you can browse in the snapshots of the backup data and copy
the files and dirs with linux commands.

The mountpoint for the backup storage is a full path (it will be created
when selcting to mount in the restore).

`restic_mountpoint = /mnt/restore`

## Cleanup options ##

### Duplicity ###

Duplicity can set a value how long to keep backup data on the backup storage.
It will keep all files of incremental and full backup to be able to restore 
data of the given min. availability.

Example if you defined a monthly full backup and keep backups for 6 monthes ... 
then it keeps all files up to 6 monthes plus all older files up to the next
full backup. You will find backup files up to 7 monthes.

```text
  1 month    (for monthly full backup)
+ 6 monthes  (min. availability for restore)
---------------------------------------------
  7 monthes  
```

The value duplicity_keep is a number and a letter for the time format.

s, m, h, D, W, M, or Y (indicating seconds, minutes, hours, days, weeks, months, or years respectively)

`duplicity_keep = 6M`

### Restic ###

Restic works like a repository and uses deduplication. The first run of a file backup
is a full backup and all following runs make an incremental backup.

There are 2 simple cleanup variants for restic forget:

* The value for "keep-last" never deletes the last N most recent snapshots.
* "restic_keep-within" keeps all created snapshots within a given period (eg. 1y5m7d2h) 

If you need more control and want less snapshots as older they are:

You have the possibility to keep all younger backups and have the possibility to delete
older backups but to keep N weekly, monthly and yearly snapshots.

Multiple values are allowed and work like an OR condition.
See the docs for more details to the keep values of the forget command <https://restic.readthedocs.io/en/latest/060_forget.html>

```text
# prune 
restic_keep-within = 6m
# restic_keep-last = 5
# restic_keep-hourly = 10
# restic_keep-daily = 30
# restic_keep-weekly = 12
# restic_keep-monthly = 12
# restic_keep-yearly = 10
```

Supported values are:

* restic_keep-last [N] - keep the last n snapshots
* restic_keep-(hourly|daily|weekly|monthly|yearly) [N] - keep then last n snapshots of a period
* restic_keep-within [duration] - keep snapshots that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
* restic_keep-within-(hourly|daily|weekly|monthly|yearly) [duration] - keep snapshots of given period that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
* restic_keep-tag [taglist] - keep snapshots with this taglist (can be specified only one in imlbackup) (default [])

`restic_keep-within = 6m`