#!/bin/bash # ================================================================================ # # JOBHELPER # helper script to share functions for parsing and handlinmg backup jobs # # -------------------------------------------------------------------------------- # ah - Axel Hahn <axel.hahn@unibe.ch> # ds - Daniel Schueler <daniel.schueler@iml.unibe.ch> # # 2016-11-10 ah,ds v1.0 # 2017-01-23 ah,ds v1.1 added j_getLastBackupAge # 2017-02-16 ah,ds v1.2 added storage helper function # 2018-02-13 ah,ds v1.3 detect samba shares based on a flag # 2022-10-07 ah v1.4 unescape regex with space to prevent "grep: warning: stray \ before white space" # 2023-03-17 ah v1.5 ignore required user on MS windows; source jobs/env if it exists; use varaiable FQDN # 2023-04-12 ah v1.6 add desktop notification # 2023-10-06 v1.7 source jobs/env_defaults # ================================================================================ # ---------------------------------------------------------------------- # CONFIG # ---------------------------------------------------------------------- DIR_SELF=$( dirname "$0" ) DIR_JOBS="${DIR_SELF}/jobs" DIR_LOGS="${DIR_SELF}/logs" JOBFILE="${DIR_JOBS}/backup.job" DIRFILE="${DIR_JOBS}/dirs.job" STORAGEFILE="${DIR_JOBS}/transfer.job" # ---------------------------------------------------------------------- # METHODS # ---------------------------------------------------------------------- # ------------------------------------------------------------ # init function # ------------------------------------------------------------ function j_init(){ j_banner . "${DIR_JOBS}/env_defaults" if [ -r "${DIR_JOBS}/env" ]; then echo "INFO: loading custom environment ${DIR_JOBS}/env" . "${DIR_JOBS}/env" fi _getFqdn if [ ! -d "${DIR_LOGS}" ]; then mkdir -p "${DIR_LOGS}" && echo "INFO: dir created ${DIR_LOGS}" fi if [ ! -d "${DIR_JOBS}" ]; then # mkdir -p ${DIR_JOBS} && echo "INFO: dir created ${DIR_JOBS}" >&2 echo "ERROR: missing jobs directory. Aborting." exit 1 fi for myfile in "${JOBFILE}" "${DIRFILE}" "${STORAGEFILE}" do if [ ! -f "${myfile}" ]; then echo "WARNING: missing a config file: $myfile" # exit 1 fi done # for date definitions like weekdays JOBDEF_LANG=$(_j_getvar "${JOBFILE}" "lang") if [ -z "$JOBDEF_LANG" ]; then JOBDEF_LANG="en_us" fi export LANG=$JOBDEF_LANG j_requireBinary "cut" j_requireBinary "date" j_requireBinary "grep" j_requireBinary "sleep" j_requireBinary "tee" j_requireBinary "touch" # for notify-send in j_notify() if [ -n "$SUDO_USER" ]; then export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u $SUDO_USER)/bus fi # j_read } # ------------------------------------------------------------ # draw an ascii banner # ------------------------------------------------------------ function j_banner(){ cat << eofbanner ___ ___ ___ ___ _______ __ | | Y | | | _ .---.-.----| |--.--.--.-----. |. |. |. | |. 1 | _ | __| <| | | _ | |. |. \_/ |. |___ |. _ |___._|____|__|__|_____| __| |: |: | |: 1 | |: 1 \ |__| |::.|::.|:. |::.. . | |::.. . / \`---\`--- ---\`-------' \`-------' eofbanner sed "s#^# #g" "$0.banner" 2>/dev/null || echo $0 echo '_______________________________________________________________________________ ___ __ _' echo } # ------------------------------------------------------------ # get list of all directories to backup / restore # ------------------------------------------------------------ function j_getDirs2Backup(){ STORAGE_SAMBASHARES=$(_j_getvar "${STORAGEFILE}" "sambashares") _j_getvar "${JOBFILE}" dir-localdumps _j_getvar "${JOBFILE}" dir-dbarchive _j_getvar "${DIRFILE}" include # get dirs of filesets, i.e. # set-custom-[key]--dir = /home/ladmin _j_getvar "${DIRFILE}" "set.*--dir" # detect Samba shares (set sambashares = 1 for it) if [ -z "${STORAGE_SAMBASHARES}" ] || [ "${STORAGE_SAMBASHARES}" -eq 0 ]; then echo NO >/dev/null else if [ -f /etc/samba/smb.conf ]; then for dirOfShare in $( grep "path.*=" "/etc/samba/smb.conf" | grep -v "#.*path" | cut -f 2 -d "=" ) do echo "$dirOfShare" done fi fi } # ------------------------------------------------------------ # get the id of a given backup dir (to find includes and # excludes for it) # param string path # ------------------------------------------------------------ function j_getSetnameOfPath(){ grep "^set.*dir = $*$" "${DIRFILE}" | cut -f 1 -d "=" | sed "s#\-\-dir ##g" } # ------------------------------------------------------------ # get full target of the current host with a given # backed up directory # param string name of directory # param string name of host (for future releases) # ------------------------------------------------------------ function j_getFullTarget(){ sTmpSafeName=$(j_getSafename "$1") sTmpHostname=$2 if [ -z "$sTmpHostname" ]; then sTmpHostname=$FQDN fi if [ -z "${STORAGE_BASEDIR}" ]; then STORAGE_BASEDIR=$(_j_getvar "${STORAGEFILE}" "storage") fi echo "${STORAGE_BASEDIR}/${sTmpHostname}/${sTmpSafeName}" } # ------------------------------------------------------------ # replace / to _ to get a save filename for a directory to # backup or restore # param string name of directory # ------------------------------------------------------------ function j_getSafename(){ echo "${*//[\/\\:]/_}" } # ------------------------------------------------------------ # get a value from a job config file # param string name of jobfile # param string name of variable to fetch value from # ------------------------------------------------------------ function _j_getvar(){ if [ ! -r "${1}" ]; then >&2 echo "ERROR: cannot read file: ${1}. Abort." exit 100 fi grep "^${2} = " < "${1}"| cut -f 3- -d " " } # ------------------------------------------------------------ # read local jobdescription and set as variables # ------------------------------------------------------------ # ------------------------------------------------------------ # execute hook skripts in a given directory in alphabetic order # param string name of hook directory # param string optional: integer of existcode or "" for non-on-result hook # ------------------------------------------------------------ function _j_runHooks(){ local _hookbase="$1" local _exitcode="$2" local _hookdir="$( dirname $0 )/hooks/$_hookbase" if [ -z "$_exitcode" ]; then _hookdir="$_hookdir/always" elif [ "$_exitcode" = "0" ]; then _hookdir="$_hookdir/on-ok" else _hookdir="$_hookdir/on-error" fi for hookscript in $( ls -1a "$_hookdir" | grep -v "^\." | sort ) do if [ -x "$_hookdir/$hookscript" ]; then h3 "HOOK: start $_hookdir/$hookscript ..." $_hookdir/$hookscript else h3 "HOOK: SKIP $_hookdir/$hookscript (not executable) ..." fi done # if an exitcode was given as param then run hooks without exitcode # (in subdir "always") if [ -n "$_exitcode" ]; then _j_runHooks "$_hookbase" fi echo } # ------------------------------------------------------------ # parse day of week and day of month and echo 0 or 1 # # full = DOW:Mon,Tue,Wed # full = DOM:07,08,09, # # It is used for inc = ... and full = ... # # param string date to compare # param string value of full|inc in backup.job # ------------------------------------------------------------ function _j_runToday(){ typeset -i local bToday=0 local sCompDate="$1" shift 1 local value="$*" # grep weekday echo "$value" | grep "^DOW:" | grep $(date +%a -d "$sCompDate") >/dev/null && bToday=1 # grep day of month echo "$value" | grep "^DOM:" | grep $(date +%d -d "$sCompDate") >/dev/null && bToday=1 # grep nth weekday of a month echo "$value" | grep "^WDM:" >/dev/null if [ $? -eq 0 ]; then typeset -i local iDayOfMonth iDayOfMonth=$(date +%e -d "$sCompDate") typeset -i local iWeekday iWeekday=$(date +%u -d "$sCompDate") # `date +%u` - weekday as int; Sun = 0 # `date +%e` - day in date typeset -i local iWeekInMonth iWeekInMonth=$(echo $(( ( ${iDayOfMonth} - ${iWeekday} + 6 ) / 7 )) ) typeset -i local n n=$(echo "$value" | grep "^WDM:" | cut -f 2- -d ":" | cut -c 1) local sDay sDay=$(echo "$value" | grep "^WDM:" | cut -f 2- -d ":" | cut -f 2 -d " ") if [ ${n} -eq ${iWeekInMonth} ] && [ ${sDay} = $(date +%a -d "$sCompDate") ]; then bToday=1 fi fi echo $bToday } # ------------------------------------------------------------ # parse day of week and day of month and echo 0 or 1 # # full = DOW:Mon,Tue,Wed # full = DOM:07,08,09, # # It is used for inc = ... and full = ... # # param string value of full|inc in backup.job # ------------------------------------------------------------ function _j_isToday(){ sCmpDate=$(date +%s) _j_runToday "@$sCmpDate" "$*" } # ------------------------------------------------------------ # read local jobdescription and set as variables # ------------------------------------------------------------ function j_read(){ # mytime=$(date +%H%M) # --- parse something BACKUP_TARGETDIR=$(_j_getvar "${JOBFILE}" "dir-local-dumps") export BACKUP_TARGETDIR JOBDEF_TYPE=$(_j_getvar ${STORAGEFILE} "type") export JOBDEF_TYPE if [ "$JOBDEF_TYPE" != "auto" ]; then # ----- detect if current date matches a definition "full = ..." local cfg_full; cfg_full=$(_j_getvar "${STORAGEFILE}" "full") local bIsTodayFull; bIsTodayFull=$(_j_isToday "$cfg_full") # ... if "1" ... then verify time with "start-time-full = ..."" if [ $bIsTodayFull -eq 1 ]; then local sStart sStart=$(_j_getvar "${STORAGEFILE}" "start-time-full") test -z "$sStart}" && sStart="." if date +%H:%M | grep "$sStart" >/dev/null; then JOBDEF_TYPE=full fi fi fi if [ -z "$JOBDEF_TYPE" ]; then JOBDEF_TYPE=auto fi test "${JOBDEF_TYPE}" = "auto" && JOBDEF_AUTO=$(_j_getvar ${STORAGEFILE} "auto") export JOBDEF_AUTO _j_setLogfile } # ------------------------------------------------------------ # read local jobdescription and set as variables # ------------------------------------------------------------ function _j_setLogfile(){ JOB_DONEFILE=${DIR_LOGS}/${JOBDEF_TYPE}-$(date +%Y%m%d-%H%M%S) JOB_LOGFILE="${JOB_DONEFILE}.log" export JOB_LOGFILE } # ------------------------------------------------------------ # check if a binary exists - and abort if not # param string name of file # param bool flag to skip; default: none = abort on miss # ------------------------------------------------------------ function j_requireBinary(){ # echo "CHECK binary $1" which "$1" >/dev/null 2>&1 rcself=$? if [ $rcself -ne 0 ]; then rc=$rc+$rcself echo "INFO: missing binary $1" if [ -z "$2" ]; then exit 3 fi return 1 fi return 0 } # ------------------------------------------------------------ # check if a process name exisats # param string regex to find in ps -ef output # param bool flag to skip; default: none = abort on miss # ------------------------------------------------------------ function j_requireProcess(){ # echo "CHECK process $1" # ps -ef | grep -v grep | grep -E "$1" >/dev/null pgrep -l "$1" >/dev/null rcself=$? if [ $rcself -ne 0 ]; then rc=$rc+$rcself echo "INFO: missing process $1" if [ -z "$2" ]; then exit 4 fi return 1 fi return 0 } # detect ms windows by cygwin, mingw, msys ... # see https://en.wikipedia.org/wiki/Uname # return code is 0 for YES function _isMswindows(){ uname | grep -iE "(CYGWIN_NT|MINGW|MSYS_NT|Windows_NT|WindowsNT)" >/dev/null } # get fqdn and put it into FQDN; called in j_init # - with gnu core utils: hostname -f # - on ms windows: grep from output of ipconfig and %COMPUTERNAME% function _getFqdn(){ if [ -z "$FQDN" ]; then if ! FQDN=$( hostname -f 2>/dev/null ); then echo "INFO: hostname -f is not available" if _isMswindows; then if [ -n "$COMPUTERNAME" ]; then local _domain _domain=$( ipconfig -all | grep "Primary Dns" | cut -f 2 -d ':' | tr -d ' ' ) FQDN="${COMPUTERNAME}.${_domain}" fi fi fi if [ -z "$FQDN" ]; then echo "ERROR: unable to detect fqdn. Aborting." exit 1 fi fi # TODO force fqdn? # if ! echo "$FQDN" | grep "\..*" >/dev/null; then # echo "ERROR: hostname [$FQDN] is not a fqdn and does not contain a domain name. Aborting." # exit 1 # fi # echo "INFO: FQDN is [$FQDN]" } # show a desktop notification using notify-send # param string summary (aka title) # param string message text # paran integer optional: exitcode; if set it adds a prefix OK or ERRROR on summary and sets urgency on error function j_notify(){ local _summary="IML BACKUP :: $1" local _body="$( date +%H:%M:%S ) $2" local _rc="$3" if which notify-send >/dev/null 2>&1; then if [ -n "$DBUS_SESSION_BUS_ADDRESS" ]; then local _urgency="normal" if [ -n "$_rc" ]; then if [ "$_rc" = "0" ]; then _summary="OK: ${_summary}" else _summary="ERROR: ${_summary}" _urgency="critical" fi fi su "$SUDO_USER" -c "notify-send --urgency=${_urgency} '${_summary}' '${_body}'" fi fi } # ------------------------------------------------------------ # check if it was startet with a given user # This is skipped if MS windows was detected with "mingw". # param string username, i.e. root # ------------------------------------------------------------ function j_requireUser(){ if _isMswindows; then echo "SKIP: j_requireUser $1 is not handled on MS Windows." else sUser=$(id | cut -f 2 -d "(" | cut -f 1 -d ")") if [[ "$sUser" != "$1" ]]; then >&2 echo "ERROR: user $1 is reqired." exit 5 fi fi } # ---------------------------------------------------------------------- # INIT # ---------------------------------------------------------------------- j_init