Skip to content
Snippets Groups Projects
Commit 7b4e92ba authored by Hahn Axel (hahn)'s avatar Hahn Axel (hahn)
Browse files

Merge branch '5046-finetune-restic-params' into 'master'

5046 finetune restic params

See merge request !38
parents 8d0caff6 199a39f0
No related branches found
No related tags found
1 merge request!385046 finetune restic params
...@@ -169,17 +169,36 @@ s, m, h, D, W, M, or Y (indicating seconds, minutes, hours, days, weeks, months, ...@@ -169,17 +169,36 @@ s, m, h, D, W, M, or Y (indicating seconds, minutes, hours, days, weeks, months,
Restic works like a repository and uses deduplication. The first run of a file backup Restic works like a repository and uses deduplication. The first run of a file backup
is a full backup and all following runs make an incremental backup. is a full backup and all following runs make an incremental backup.
There are 2 simple cleanup variants for restic forget:
* The value for "keep-last" never deletes the last N most recent snapshots.
* "restic_keep-within" keeps all created snapshots within a given period (eg. 1y5m7d2h)
If you need more control and want less snapshots as older they are:
You have the possibility to keep all younger backups and have the possibility to delete You have the possibility to keep all younger backups and have the possibility to delete
older backups but to keep N weekly, monthly and yearly snapshots. older backups but to keep N weekly, monthly and yearly snapshots.
The value for "keep-last" never deletes the last N most recent snapshots. Multiple values are allowed and work like an OR condition.
See the docs for more details to the keep values of the forget command <https://restic.readthedocs.io/en/latest/060_forget.html>
```text ```text
# prune # prune
restic_keep-last = 5 restic_keep-within = 6m
restic_keep-hourly = 100 # restic_keep-last = 5
restic_keep-daily = 90 # restic_keep-hourly = 10
restic_keep-weekly = 12 # restic_keep-daily = 30
restic_keep-monthly = 12 # restic_keep-weekly = 12
restic_keep-yearly = 10 # restic_keep-monthly = 12
# restic_keep-yearly = 10
``` ```
Supported values are:
* restic_keep-last [N] - keep the last n snapshots
* restic_keep-(hourly|daily|weekly|monthly|yearly) [N] - keep then last n snapshots of a period
* restic_keep-within [duration] - keep snapshots that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
* restic_keep-within-(hourly|daily|weekly|monthly|yearly) [duration] - keep snapshots of given period that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
* restic_keep-tag [taglist] - keep snapshots with this taglist (can be specified only one in imlbackup) (default [])
`restic_keep-within = 6m`
\ No newline at end of file
...@@ -255,13 +255,26 @@ restic_tag = imlbackup ...@@ -255,13 +255,26 @@ restic_tag = imlbackup
restic_mountpoint = /mnt/restore restic_mountpoint = /mnt/restore
# prune # prune
restic_keep-hourly = 100 restic_keep-within = 6m
restic_keep-daily = 90 # restic_keep-last = 5
restic_keep-weekly = 12 # restic_keep-hourly = 10
restic_keep-monthly = 12 # restic_keep-daily = 30
restic_keep-yearly = 10 # restic_keep-weekly = 12
# restic_keep-monthly = 12
# restic_keep-yearly = 10
``` ```
For pruning with restic forget the following values can be used.
Multiple values are allowed and work like an OR condition.
See the docs for more details to the keep values of the forget command <https://restic.readthedocs.io/en/latest/060_forget.html>
* restic_keep-last [N] - keep the last n snapshots
* restic_keep-(hourly|daily|weekly|monthly|yearly) [N] - keep then last n snapshots of a period
* restic_keep-within [duration] - keep snapshots that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
* restic_keep-within-(hourly|daily|weekly|monthly|yearly) [duration] - keep snapshots of given period that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
* restic_keep-tag [taglist] - keep snapshots with this taglist (can be specified only one in imlbackup) (default [])
## restore-path ## restore-path
Default path where to put restored file when running *restore.sh*. Default path where to put restored file when running *restore.sh*.
......
...@@ -147,11 +147,12 @@ restic_tag = imlbackup ...@@ -147,11 +147,12 @@ restic_tag = imlbackup
restic_mountpoint = /mnt/restore restic_mountpoint = /mnt/restore
# prune # prune
restic_keep-last = 5 restic_keep-within = 6m
restic_keep-hourly = 100 # restic_keep-last = 5
restic_keep-daily = 90 # restic_keep-hourly = 10
restic_keep-weekly = 12 # restic_keep-daily = 30
restic_keep-monthly = 12 # restic_keep-weekly = 12
restic_keep-yearly = 10 # restic_keep-monthly = 12
# restic_keep-yearly = 10
# ---------------------------------------------------------------------- # ----------------------------------------------------------------------
...@@ -9,7 +9,8 @@ ...@@ -9,7 +9,8 @@
# ah - Axel Hahn <axel.hahn@iml.unibe.ch> # ah - Axel Hahn <axel.hahn@iml.unibe.ch>
# 2021-05-19 ah v0.1 INIT ... WIP # 2021-05-19 ah v0.1 INIT ... WIP
# 2022-01-06 ah v0.2 added support for Repository with REST and authentication # 2022-01-06 ah v0.2 added support for Repository with REST and authentication
# 2022-02-09 ah v0.3 show difff to last backup; update pruning # 2022-02-09 ah v0.3 show diff to last backup; update pruning
# 2022-02-09 ah v0.3 update pruning; more keep-params
# ================================================================================ # ================================================================================
# -------------------------------------------------------------------------------- # --------------------------------------------------------------------------------
...@@ -153,10 +154,11 @@ ...@@ -153,10 +154,11 @@
${ARGS_DEFAULT} \ ${ARGS_DEFAULT} \
--tag $_tag \ --tag $_tag \
--group-by paths,tags \ --group-by paths,tags \
--prune \
--cleanup-cache" --cleanup-cache"
local _keep local _keep
for mykeep in last hourly daily weekly monthly yearly for mykeep in last hourly daily weekly monthly yearly within within-hourly within-daily within-weekly within-monthly within-yearly tag
do do
_keep=$( _j_getvar "${STORAGEFILE}" "${CFGPREFIX}keep-${mykeep}") _keep=$( _j_getvar "${STORAGEFILE}" "${CFGPREFIX}keep-${mykeep}")
if [ -n "$_keep" ]; then if [ -n "$_keep" ]; then
......
...@@ -29,6 +29,7 @@ ...@@ -29,6 +29,7 @@
# 2021-01-29 ah,ds, v1.12 abort on empty passphrase # 2021-01-29 ah,ds, v1.12 abort on empty passphrase
# 2021-05-19 ah,ds, v2.0 plugin driven to support multiple backup tools (duplicity + restic) # 2021-05-19 ah,ds, v2.0 plugin driven to support multiple backup tools (duplicity + restic)
# 2021-12-02 ah v2.1 added parameter "prune" to cleanup only # 2021-12-02 ah v2.1 added parameter "prune" to cleanup only
# 2022-02-10 ah v2.2 update logging (removing tee)
# ================================================================================ # ================================================================================
...@@ -130,14 +131,16 @@ ...@@ -130,14 +131,16 @@
if [ "$1" = "prune" ]; then if [ "$1" = "prune" ]; then
doBackup=0 doBackup=0
fi fi
exec 1> >( tee -a "$transferlog" ) 2>&1
echo "INFO: Start logging into $transferlog"
h1 `date` TRANSFER LOCAL DATA TO STORAGE | tee -a $transferlog h1 `date` TRANSFER LOCAL DATA TO STORAGE
echo "METHOD : $METHOD" | tee -a $transferlog echo "METHOD : $METHOD"
echo "TARGET : ${STORAGE_BASEDIR}" | tee -a $transferlog echo "TARGET : ${STORAGE_BASEDIR}"
echo "REGISTER : ${STORAGE_REGISTER}" | tee -a $transferlog echo "REGISTER : ${STORAGE_REGISTER}"
echo "TOOL : $STORAGE_BIN" | tee -a $transferlog echo "TOOL : $STORAGE_BIN"
echo | tee -a $transferlog echo
. `dirname $0`/plugins/transfer/$STORAGE_BIN.sh || exit 1 . `dirname $0`/plugins/transfer/$STORAGE_BIN.sh || exit 1
test -z "$STORAGE_REGISTER" || . `dirname $0`/plugins/register/$STORAGE_REGISTER.sh || exit 1 test -z "$STORAGE_REGISTER" || . `dirname $0`/plugins/register/$STORAGE_REGISTER.sh || exit 1
...@@ -242,7 +245,7 @@ ...@@ -242,7 +245,7 @@
sleep $iRnd sleep $iRnd
fi fi
done done
fi | tee -a $transferlog fi
h2 "`date` PRE transfer tasks" h2 "`date` PRE transfer tasks"
...@@ -345,7 +348,7 @@ ...@@ -345,7 +348,7 @@
echo $rc > ${rcfile} echo $rc > ${rcfile}
exit $rc exit $rc
) | tee -a $transferlog )
# rc=${PIPESTATUS[0]} # rc=${PIPESTATUS[0]}
rc=`cat ${rcfile}` rc=`cat ${rcfile}`
...@@ -364,22 +367,22 @@ ...@@ -364,22 +367,22 @@
echo "SKIP" echo "SKIP"
else else
unregisterBackupSlot `hostname -f` $rc unregisterBackupSlot `hostname -f` $rc
fi | tee -a $transferlog fi
h2 "`date` Backup finished" h2 "`date` Backup finished"
echo STATUS $0 exit with final returncode rc=$rc | tee -a $transferlog echo STATUS $0 exit with final returncode rc=$rc
echo | tee -a $transferlog echo
if [ $rc -eq 0 ]; then if [ $rc -eq 0 ]; then
color ok color ok
echo Backup OK | tee -a $transferlog echo Backup OK
else else
color error color error
echo Backup FAILED :-/ | tee -a $transferlog echo Backup FAILED :-/
fi fi
color reset color reset
echo | tee -a $transferlog echo
typeset -i TIMER_TRANSFER=`date +%s`-$TIMER_TRANSFER_START typeset -i TIMER_TRANSFER=`date +%s`-$TIMER_TRANSFER_START
echo `date` TRANSFER DONE in $TIMER_TRANSFER sec | tee -a $transferlog echo `date` TRANSFER DONE in $TIMER_TRANSFER sec
ls -l $transferlog ls -l $transferlog
exit $rc exit $rc
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment