# storage_helper.sh
The script should be alwys started with the non privileged user (imlbackup).
Without a parameter you get a help:
```txt
imlbackup@backup-host:~$ ./storage_helper.sh
---------- :: STORAGE :: storage-connector :: ----------
SYNTAX:
storage_helper.sh [function]
backupstatus
show all servers and their backup times
register [hostname]
add a slot for a backup client
unregister [hostname] [statuscode]
remove slot for a backup client.
The statuscode is an integer of the return code.
setactive
Reactivate an inctave backup repo.
setinactive
Mark a backup repo as inactive.
This prevents errors of missing backup data
if a host is obsolete and was shut down.
status
show current reserved slots
usage [full|[filter to hostname]] - DEPRECATED
show used diskspace
you can set a string to show additionally details if these hosts
if you add "full" then details of all hosts will be shown
```
## What backup clients do
Before a backup client send its data it wants to register.
`./storage_helper.sh register [hostname]`
The exitcode is 0 if it got a free slot. And non-zero if all slots are in use (see iMaxConnections in config). The backup client will wait a random time until it gets a free slot to write its data.
If the backup is finished it sends
`./storage_helper.sh unregister [hostname]`
## Admin tasks
### See current activity
```txt
$ ./storage_helper.sh status
---------- :: STORAGE :: storage-connector :: ----------
STATUS: 0 existing connection(s). Allowed maximum is 10.
```
### Backup status
You get a table of all servers that sent backup data.
There are listed
* last backup with start time, end time, duration, existcode
* age of the last backup (to see if a system does not send data anymore)
* size of backup data
* Each line per server is colored depending on backup status or time of last sent data
```txt
$ ./storage_helper.sh backupstatus
---------- :: STORAGE :: storage-connector :: ----------
STATUS: 0 existing connection(s). Allowed maximum is 10.
This table shows the time and duration [s] of the last backup for each server.
server | start | end | duration | rc | age [h] | size
-----------------------------------------------------------------------------------------------------------------------------
. srv1.example.com | 2022-09-19 23:04:14 | 2022-09-19 23:17:52 | 818 | 0 | 12 | 951G /netshare/restic-backup/srv1.example.com
. srv2.example.com | 2022-09-19 23:03:06 | 2022-09-19 23:18:28 | 922 | 0 | 12 | 955G /netshare/restic-backup/srv2.example.com
...
Legend
. OK | ? not started | R running | D disabled | E error
total : 26 servers
errors: 0
rc=0
```
### deactivate a backup folder
If you delete a host but must keep its backup data then you can set it inactive. Then you don't get an error because of missing incoming data.
To disable a backup target:
```txt
$ ./storage_helper.sh setinactive
---------- :: STORAGE :: storage-connector :: ----------
--- list of backup repositories
/netshare/restic-backup/srv1.example.com (active)
/netshare/restic-backup/srv1.example.com (active)
...
Repo to deactivate >
```
Enter the full path here, eg. `/netshare/restic-backup/srv1.example.com` and Return.
Then comes an optional input for a hint - that is just for you and your team. Leave a message if the host is deprecated, the application was moved etc.
In the backup status the deactivated target is listed with D = Disabled and is gray.
BTW: the opposite way is possible to:
`./storage_helper.sh setactive`
# rest_pruner.sh
**!! This script is in version 0.1 - and work in progress !!**
The pruner script is for restic rest server with append only option. It can prune all repositories on server side.
## inc_config.sh
The inc_config.sh mus contain 3 variables for pruning
```bash
# for prune on restic rest server
prune_basedir=/netshare/restic-backup
prune_params="--group-by paths,tags --prune --keep-within 180d --max-unused unlimited --max-repack-size 10G --cleanup-cache"
prune_skipdays="7"
prune_cachedir=${prune_basedir}/.cache_for_pruning
```
## rest_pruner.cfg
To access the different local repositories we need the RESTIC_PASSWORD for each repository.
The config file `rest_pruner.cfg` contains lines in the syntax
`<USER>:<RESTIC_PASSWORD>`
If a directory matches `${prune_basedir}/<USER>` then it will be pruned.
You need a mechanism to create this file eg. by Ansible.
For securiy reasons this file must be owned by root:root and must have the permissions 0400.
```txt
ls -l rest_pruner.cfg
-r--------. 1 root root 159 Feb 1 13:35 rest_pruner.cfg
```
## Syntax
```txt
------------------------------------------------------------------------------
--------========###| RESTIC REST PRUNER :: v0.4 |###=======--------
------------------------------------------------------------------------------
Pruner for restic rest server with append only option.
This script prunes all repositories on server side.
The config file [rest_pruner.cfg] contains <USER>:<RESTIC_PASSWORD>
If a directory matches /netshare/restic-backup/<USER> then it will be pruned.
Institute for Medical Education * University of Bern
GNU GPL 3.0
SYNTAX:
rest_pruner.sh [OPTIONS] [FILTER]
OPTIONS:
-h, --help show help and exit.
-a, --all process all repositories, default: rest only + archives
-d, --debug show more infos
-f, --force force pruning; do not wait 7 days
-s, --status show pruning status with last prunes
PARAMETERS:
FILTER regex to filter directory list in
/netshare/restic-backup/*
EXAMPLES:
rest_pruner.sh
Start pruning of all matching repositories
rest_pruner.sh mail
Prune servers that match "mail",
eg. my-mailhub.example.com
rest_pruner.sh --force mail
Prune servers that match "mail",
eg. my-mailhub.example.com
rest_pruner.sh --status
Show statistics with last prunes, if it is archive or
running or on error.
It shows repositories with passwords and archives.
Other repositories are not visible until you
specify --all.
```
## How does it work
It detects some requirements:
* was it started by root? (The help is shown without being root)
* permissions of rest_pruner.cfg
* variables from inc_config.sh
It loops over the starting dir `${prune_basedir}` and reads all its subdirectories.
If a subdir matches a configuration entry (text before first ":" `<USER>:<RESTIC_PASSWORD>`) then the prune process will be started. It detects the username of the owner of the directory and executes
`su - $_user - /bin/bash -c "<restic forget ...>"`
Exitcodes:
* 0 if the run was without errors.
* >0 if a prune run of a reposaitory failed.
* 1..3 if a requirement failed to start
The output of the prune command is written into
`_last_prune/<USER>.log`
If a job failed it is renamed to
`_last_prune/<USER>.log.error`