Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
I
iml-backup
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
IML Open Source
iml-backup
Commits
312fbe37
Commit
312fbe37
authored
3 years ago
by
Hahn Axel (hahn)
Browse files
Options
Downloads
Patches
Plain Diff
continue class like functions
parent
1f3f8350
Branches
Branches containing commit
No related tags found
1 merge request
!43
Update localdump to classlike functions
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
localdump.sh
+95
-32
95 additions, 32 deletions
localdump.sh
plugins/localdump/mysql.sh
+57
-32
57 additions, 32 deletions
plugins/localdump/mysql.sh
with
152 additions
and
64 deletions
localdump.sh
+
95
−
32
View file @
312fbe37
...
@@ -13,9 +13,17 @@
...
@@ -13,9 +13,17 @@
# 2018-02-09 ..... fix: restore-selection of target uses default on return only
# 2018-02-09 ..... fix: restore-selection of target uses default on return only
# 2021-05-18 ..... move supported backup types to plugins/localdump/[service].sh
# 2021-05-18 ..... move supported backup types to plugins/localdump/[service].sh
# 2021-07-13 ..... remove leading ./ in localdump.sh restore
# 2021-07-13 ..... remove leading ./ in localdump.sh restore
# 2022-02-18 ..... WIP: use class like functions
# ======================================================================
# ======================================================================
# --- variables:
# BACKUP_BASEDIR {string} base directory for db dumps
# BACKUP_DATE {string} string with current timestamp; will be part of filename for backups
# BACKUP_KEEP_DAYS {int} count of days how long to keep db dumps below $BACKUP_BASEDIR
# BACKUP_PLUGINDIR {string} scripts for supported databases; [APP]/plugins/localdump
# BACKUP_SCRIPT {string} script name of db service
# BACKUP_TARGETDIR {string} target directory db dumps of current service
# SERVICENAME {string} name of db service (one of mysql|pgsql|...)
# ----------------------------------------------------------------------
# ----------------------------------------------------------------------
# CONFIG VARS
# CONFIG VARS
...
@@ -24,9 +32,9 @@
...
@@ -24,9 +32,9 @@
.
`
dirname
$0
`
/jobhelper.sh
.
`
dirname
$0
`
/jobhelper.sh
.
`
dirname
$0
`
/inc_bash.sh
.
`
dirname
$0
`
/inc_bash.sh
if
[
-r
~/.backup.conf
]
;
then
#
if [ -r ~/.backup.conf ]; then
.
~/.backup.conf
#
. ~/.backup.conf
fi
#
fi
if
[
!
-r
"
${
JOBFILE
}
"
]
;
then
if
[
!
-r
"
${
JOBFILE
}
"
]
;
then
color error
color error
...
@@ -35,30 +43,63 @@
...
@@ -35,30 +43,63 @@
exit
1
exit
1
fi
fi
BACKUP_BASEDIR
=
`
_j_getvar
${
JOBFILE
}
"dir-localdumps"
`
BACKUP_BASEDIR
=
BACKUP_PLUGINDIR
=
`
dirname
$0
`
/plugins/localdump
BACKUP_PLUGINDIR
=
# check
# Cleanup local dumps older N days
if
[
-z
"
$BACKUP_BASEDIR
"
]
;
then
typeset
-i
BACKUP_KEEP_DAYS
=
0
color error
echo
ERROR: missing config
for
backup target.
echo
There must be an entry dir-localdumps
in
${
JOBFILE
}
color reset
exit
1
fi
# CLEANUP AFTER N DAYS...
BACKUP_DATE
=
typeset
-i
BACKUP_KEEP_DAYS
=
`
_j_getvar
${
JOBFILE
}
"keep-days"
`
if
[
$BACKUP_KEEP_DAYS
-eq
0
]
;
then
# ----------------------------------------------------------------------
BACKUP_KEEP_DAYS
=
7
# FUNCTIONS 4 DB-WRAPPER
fi
# ----------------------------------------------------------------------
function
db.init
(){
BACKUP_BASEDIR
=
`
_j_getvar
${
JOBFILE
}
"dir-localdumps"
`
# ----- additional vars
# check
if
[
-z
"
$BACKUP_BASEDIR
"
]
;
then
color error
echo
ERROR: missing config
for
backup target.
echo
There must be an entry dir-localdumps
in
${
JOBFILE
}
color reset
exit
1
fi
BACKUP_PLUGINDIR
=
`
dirname
$0
`
/plugins/localdump
BACKUP_KEEP_DAYS
=
`
_j_getvar
${
JOBFILE
}
"keep-days"
`
BACKUP_DATE
=
$(
/bin/date +%Y%m%d-%H%M
)
if
[
$BACKUP_KEEP_DAYS
-eq
0
]
;
then
BACKUP_KEEP_DAYS
=
7
fi
BACKUP_DATE
=
$(
/bin/date +%Y%m%d-%H%M
)
}
# helpfer function for SERVICENAME.backup
# it is called after the service specific dump was done.
# param {string} filename of created dump file
function
db._compressDumpfile
(){
local
_outfile
=
$1
# $myrc is last returncode - set in fetchrc
if
[
$myrc
-eq
0
]
;
then
echo
-n
"gzip ... "
# compress_file "$_outfile"
echo
-n
"compressing
$1
... "
gzip
-9
-f
"
${
1
}
"
fetchrc
>
/dev/null
if
[
$myrc
-eq
0
]
;
then
cecho ok
"OK"
else
cecho error
"FAILED"
fi
else
cecho error
"ERROR occured while dumping - no gzip"
fi
ls
-l
"
$_outfile
"
*
echo
}
# ----------------------------------------------------------------------
# ----------------------------------------------------------------------
# FUNCTIONS 4 BACKUP
# FUNCTIONS 4 BACKUP
...
@@ -179,9 +220,9 @@
...
@@ -179,9 +220,9 @@
echo
You can try to restore dumps:
echo
You can try to restore dumps:
echo
"1) Restore dump files from a backup set"
echo
"1) Restore dump files from a backup set"
echo
"
`
dirname
$0
`
/restore.sh
$BACKUP_BASEDIR
"
echo
"
`
dirname
$0
`
/restore.sh
$BACKUP_BASEDIR
"
echo
"2)
Move
restored dumps into
$BACKUP_TARGETDIR
"
echo
"2)
Copy
restored dumps into
$BACKUP_TARGETDIR
"
echo
"3) Start database restore again"
echo
"3) Start database restore again"
echo
"
`
dirname
$0
`
/localdump.sh restore"
echo
"
`
dirname
$0
`
/localdump.sh restore
[service]
"
echo
echo
exit
1
exit
1
...
@@ -241,6 +282,8 @@
...
@@ -241,6 +282,8 @@
# INIT
# INIT
# ----------------------------------------------------------------------
# ----------------------------------------------------------------------
db.init
# ----- checks
# ----- checks
# . /usr/local/bin/inc_cronfunctions.sh
# . /usr/local/bin/inc_cronfunctions.sh
...
@@ -387,18 +430,38 @@
...
@@ -387,18 +430,38 @@
fi
fi
;;
;;
# ------------------------------------------------------------
# ------------------------------------------------------------
# shell)
shell
)
# . $BACKUP_SCRIPT $mode
# myshell=$( grep "^$USER" /etc/passwd | cut -f 7 -d ":" )
export
BACKUP_TARGETDIR
# echo "TYPE EXIT ... to leave $myshell"
.
$BACKUP_SCRIPT
# $myshell -i
(
# echo "Subshell was closed."
mycmd
=
# ;;
echo
echo
"Starting interactive shell..."
echo
echo
"STATUS: STILL ALPHA as long existing db plugins are not rewritten."
echo
echo
"INFO: Try
${
SERVICENAME
}
.help to see database specific commands."
echo
"INFO: Type exit and return to leave the shell."
echo
while
[
!
"
$mycmd
"
=
"exit"
]
;
do
echo
-n
"[
${
SERVICENAME
}
]"
color input
echo
-n
"
$(
pwd
)
"
color reset
echo
-n
" % "
read
-r
mycmd
if
[
!
"
$mycmd
"
=
"exit"
]
;
then
color cmd
eval
$mycmd
color reset
fi
done
)
;;
esac
esac
echo
_______________________________________________________________________________
echo
_______________________________________________________________________________
echo
STATUS
$0
exit
with final returncode
rc
=
$rc
echo
STATUS
$0
exit
with final returncode
rc
=
$rc
exit
$rc
exit
$rc
...
...
This diff is collapsed.
Click to expand it.
plugins/localdump/mysql.sh
+
57
−
32
View file @
312fbe37
#!/bin/bash
# ================================================================================
# ================================================================================
#
#
# LOCALDUMP :: MYSQL / MARIADB
# LOCALDUMP :: MYSQL / MARIADB
...
@@ -12,10 +11,11 @@
...
@@ -12,10 +11,11 @@
# 2017-03-28 ..... v1.0 added restore
# 2017-03-28 ..... v1.0 added restore
# 2022-01-20 ah v1.1 fixes with shellcheck
# 2022-01-20 ah v1.1 fixes with shellcheck
# 2022-02-14 ah v2.0 rewrite with class like functions
# 2022-02-14 ah v2.0 rewrite with class like functions
# 2022-02-18 ah v2.1 WIP: added counters
# ================================================================================
# ================================================================================
if
[
-z
"
$BACKUP_TARGETDIR
"
]
;
then
if
[
-z
"
$BACKUP_TARGETDIR
"
]
;
then
echo
"ERROR: you cannot start
$(
basename
$0
)
directly"
echo
"ERROR: you cannot start
$(
basename
$0
)
directly
. Start localdump.sh instead.
"
rc+
=
1
rc+
=
1
exit
1
exit
1
fi
fi
...
@@ -24,7 +24,15 @@ fi
...
@@ -24,7 +24,15 @@ fi
# CONFIG
# CONFIG
# --------------------------------------------------------------------------------
# --------------------------------------------------------------------------------
# flag: service was foound locally? One of 0|1
mysql_FOUND
=
0
mysql_FOUND
=
0
# counters
mysql_COUNT_CREATE
=
0
mysql_COUNT_DUMPS
=
0
mysql_COUNT_DB
=
0
mysql_COUNT_ERRORS
=
0
SOURCE_DIR
=
/var/lib/mysql
SOURCE_DIR
=
/var/lib/mysql
...
@@ -47,6 +55,7 @@ function mysql._check(){
...
@@ -47,6 +55,7 @@ function mysql._check(){
# set flag and reset return code
# set flag and reset return code
test
$rc
-eq
0
&&
mysql_FOUND
=
1
test
$rc
-eq
0
&&
mysql_FOUND
=
1
test
$rc
-eq
0
||
mysql_COUNT_ERRORS+
=
1
rc
=
0
rc
=
0
}
}
...
@@ -59,6 +68,9 @@ function mysql._check(){
...
@@ -59,6 +68,9 @@ function mysql._check(){
function
mysql.db.create
(){
function
mysql.db.create
(){
local
_dbname
=
$1
local
_dbname
=
$1
echo
"CREATE DATABASE IF NOT EXISTS
${
_dbname
}
;"
| mysql
echo
"CREATE DATABASE IF NOT EXISTS
${
_dbname
}
;"
| mysql
fetchrc
>
/dev/null
test
$myrc
-eq
0
&&
mysql_COUNT_CREATE+
=
1
test
$rc
-eq
0
||
mysql_COUNT_ERRORS+
=
1
}
}
# dump [database] --> [file]
# dump [database] --> [file]
...
@@ -67,17 +79,19 @@ function mysql.db.create(){
...
@@ -67,17 +79,19 @@ function mysql.db.create(){
# param string name of output file
# param string name of output file
function
mysql.db.dump
(){
function
mysql.db.dump
(){
local
_dbname
=
$1
local
_dbname
=
$1
local
_dumpfile
=
$2
local
_dumpfile
=
$2
mysqldump
--opt
\
mysqldump
--opt
\
--default-character-set
=
utf8
\
--default-character-set
=
utf8
\
--flush-logs
\
--flush-logs
\
--single-transaction
\
--single-transaction
\
--no-autocommit
\
--no-autocommit
\
--result-file
=
"
$_dumpfile
"
\
--result-file
=
"
$_dumpfile
"
\
"
$_dbname
"
"
$_dbname
"
fetchrc
>
/dev/null
test
$myrc
-eq
0
&&
mysql_COUNT_DUMPS+
=
1
test
$rc
-eq
0
||
mysql_COUNT_ERRORS+
=
1
}
}
# import [file] --> [database]
# import [file] --> [database]
...
@@ -88,11 +102,19 @@ function mysql.db.import(){
...
@@ -88,11 +102,19 @@ function mysql.db.import(){
local
_dumpfile
=
$1
local
_dumpfile
=
$1
local
_dbname
=
$2
local
_dbname
=
$2
zcat
"
$_dumpfile
"
| mysql
"
${
_dbname
}
"
zcat
"
$_dumpfile
"
| mysql
"
${
_dbname
}
"
fetchrc
>
/dev/null
test
$myrc
-eq
0
&&
mysql_COUNT_IMPORT+
=
1
test
$rc
-eq
0
||
mysql_COUNT_ERRORS+
=
1
}
}
# show a list of existing databases
# show a list of existing databases
function
mysql.db.list
(){
function
mysql.db.list
(){
mysql
-Ee
"show databases ;"
|
grep
"^Database:"
|
awk
'{ print $2 }'
# mysql -Ee "show databases ;" | grep "^Database:" | awk '{ print $2 }'
local
_result
=
$(
mysql
-Ee
"show databases ;"
)
fetchrc
>
/dev/null
test
$myrc
-eq
0
&&
mysql_COUNT_DB
=
$(
echo
"
$_result
"
|
grep
-c
"^Database:"
)
test
$myrc
-eq
0
&&
echo
"
$_result
"
|
grep
"^Database:"
|
awk
'{ print $2 }'
test
$rc
-eq
0
||
mysql_COUNT_ERRORS+
=
1
}
}
# --------------------------------------------------------------------------------
# --------------------------------------------------------------------------------
...
@@ -139,19 +161,8 @@ function mysql.backup(){
...
@@ -139,19 +161,8 @@ function mysql.backup(){
_outfile
=
"
${
BACKUP_TARGETDIR
}
/
$(
get_outfile
${
_dbname
}
)
.sql"
_outfile
=
"
${
BACKUP_TARGETDIR
}
/
$(
get_outfile
${
_dbname
}
)
.sql"
mysql.db.dump
"
$_dbname
"
"
$_outfile
"
mysql.db.dump
"
$_dbname
"
"
$_outfile
"
fetchrc
db._compressDumpfile
"
$_outfile
"
# $myrc is last returncode - set in fetchrc
if
[
$myrc
-eq
0
]
;
then
echo
-n
"gzip ... "
compress_file
"
$_outfile
"
else
color error
echo
"ERROR occured - no gzip"
color reset
fi
ls
-l
"
$_outfile
"
*
echo
done
done
}
}
...
@@ -169,13 +180,14 @@ function mysql.help(){
...
@@ -169,13 +180,14 @@ function mysql.help(){
# fi
# fi
# done
# done
cat
<<
EOHELP
cat
<<
EOHELP
h
elp for MYSQL-DOT functions
H
elp for MYSQL-DOT functions
(1) high level functions
(1) high level functions
mysql.check [0|1] check if mysql is available
mysql.available silent; exitcode is 0 if mysql is available
mysql.check [0|1] check if mysql is available; shows missing checks
mysql.backup backup all databases
mysql.backup backup all databases
mysql.restore FILE DBNAME
restore database
mysql.restore
[
FILE
[
DBNAME
]]
restore database
(2) functions on database level
(2) functions on database level
...
@@ -226,8 +238,21 @@ function mysql.restore(){
...
@@ -226,8 +238,21 @@ function mysql.restore(){
}
}
function
mysql.shell
(){
# WIP: show status
echo
"hi"
function
mysql.status
(){
h2
"WIP: Status"
h3
"Databases (max. 15)"
mysql.db.list |
head
-15
h3
"Counters"
cat
<<
EOSTATUS
found Dbs:
$mysql_COUNT_DB
created :
$mysql_COUNT_CREATE
dumped :
$mysql_COUNT_DUMPS
ERRORS :
$mysql_COUNT_ERRORS
EOSTATUS
}
}
# --------------------------------------------------------------------------------
# --------------------------------------------------------------------------------
# MAIN
# MAIN
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment