diff --git a/docs/30_Configuration/20_Database.md b/docs/30_Configuration/20_Database.md index 971f0168d32cffa3b6db93e81ef5f2bb103034fb..aa37f253167185540e30eb65550fded4354d5db7 100644 --- a/docs/30_Configuration/20_Database.md +++ b/docs/30_Configuration/20_Database.md @@ -1,9 +1,11 @@ -# Basic settings for database backups # +# Database backups # + +## Shared settings for all database types ## There are 2 required values in the jobs/backup.job ```text -dir-localdumps = /var/iml-backup +dir-localdumps = /var/iml-backup/backup keep-days = 7 ``` @@ -11,21 +13,139 @@ This defines the backup target for sql dumps and how long they will be kept loca There is an optional value to define the target directory for archived dumps. This value is used for couchdb2 only. -`dir-dbarchive = /var/localdumps/archive` +`dir-dbarchive = /var/iml-backup/archive` see [backup.job](50_File_backup.job.md) +## Profiles + +In the folder plugins/localdump/profiles/ are several ini files. + +They describe a database environment by the [detect] section. + +If a profile matches then a database will be dumped or restored. +For the detection you can verify + +* existing files +* running processes +* open tcp port + * localhost or a remote system + * on a local machine (if no hostname is given) you can check the process using this port + +After installation there are some ini files that contain a common + +### File names + +File convention: `<DBTYPE>_<PROFILE>.ini` + +* <DBTYPE> must match a database dumper in plugins/localdump/ without extension ".sh" +* <DBTYPE>_<PROFILE> is the target folder to store backups. + + * <PROFILE> is a custom name to + * _<PROFILE> can be empty to mark a local database of the given type. + +### Detect + +To detect the existance of you can use the following keys. If all found detect entries match it is counted as detected. + +| Key | Type | Description | +|--- |--- |--- | +| binary | string | Binary that must be found in path. Use comma to separate multiple binaries. +| file[] | string | full path of a file. It must match "type". It can be given multiple times. +| process | regex | A binary to be matched in the process list +| tcp | integer | A tcp port number that must be in use +| tcp-process | regex | A process name offering the local tcp port (regex for last column in `netstat -tulpen`). If process names can differ depending on an os use a pipe to set alternatives eg `postgres|postmaster` +| tcp-target | string | A hostname for tcp port check, eg localhost +| type | regex | a matching regex for `file -b FILE | grep -i REGEX; see "file[]"` + +Example: + +Mysql detection in a docker container: + +```ini +[detect] +process = 'mysqld|mariadb' +tcp = 13306 +tcp-target = localhost +tcp-process = 'rootlesskit' +``` + +Multiple Sqlite files + +```ini +[detect] +file[] = "/home/axel/data/docker/ciserver/data/imldeployment/data/database/logs.db" +file[] = "/home/axel/data/docker/ciserver/public_html/valuestore/data/versioncache.db" +type = "sqlite" +``` + +### Setter + +If a profile detection was successful then values in the [set] section will be applied. + +| Key | Type | Description | +|--- |--- |--- | +| su | string | User for su command; used for postgres +| dbuser | string | Database user; for replacement +| dbpassword | string | Password of database user; for replacement +| params | string | cli paramaters für dump / restore tools. +| env | string | extend environment with some variables, eg. export var1="something here". After backup/ restore this variables will be unset + +Replacements that can be used for values `params` and `env`: + +| Key | Description | +|--- |--- | +| {dbpassword} | value of [set] -> dbpassword +| {dbuser} | value of [set] -> dbuser +| {tcp-port} | value of [detect] -> tcp-port +| {tcp-target} | value of [detect] -> tcp-target + +```ini +[set] + +su = '' +dbuser = 'root' +dbpassword = '12345678' +params = '--port={tcp-port} --password={dbpassword} --user={dbuser} --host={tcp-target}' + +env = 'export var1="happy meal"; export var2="new"; export var3="year!"' + +``` + ## Backup sqlite ## Sqlite files can be located anywhere in the filesystem. That's why the -cannot be located with an auto detection. You need to define them in -the file jobs/backup-dbfiles.job first. +cannot be located with an auto detection. -Per database file set a line with the `sqlite = ` prefix +* In the detect section set `type = "sqlite"` +* Per database file set a line with the `file[] = ` prefix -```text -sqlite = /var/lib/whatever/sqlite-database_01.db -sqlite = /var/lib/somewhere/else/db.sqlite -``` +This is the plugins/localdump/profile/sqlite.ini.example: + +```ini +# ====================================================================== +# +# LOCAL SQLITE DATABASES +# +# ====================================================================== + +[detect] + +# the filetype to detect using file command +type = "sqlite" + +# list of files to backup +# file[] = "/var/www/database/logs.db" -see [backup-dbfiles.job](50_File_backup-dbfiles.job.md) + +[set] + +su = '' +dbuser = '' +dbpassword = '' + +params = '' +env = '' + +# ---------------------------------------------------------------------- +``` diff --git a/docs/40_Usage/20_Database.md b/docs/40_Usage/20_Database.md index 9b931f2bb5bb9a565f6a236524310b8b8b3ef112..74f89365a467fa90b816f024dbe764d51ba3d274 100644 --- a/docs/40_Usage/20_Database.md +++ b/docs/40_Usage/20_Database.md @@ -1,35 +1,53 @@ ## Description To create backup database dumps without transfer of local directory to a backup target use `sudo ./localdump.sh`. -Backup dumps will be stored as gzip files into `/var/iml-backup/[service]`. +Backup dumps will be stored as gzip files into `/var/iml-backup/[profile]`. ## Help ```text +LOCALDUMP detects existing local databases and dumps them locally. +It is included in the backup.sh to dump all before a file backup will store +them. It can be started seperately for manual database backups or for restore. + SYNTAX: -localdump.sh [[operation]] [Name_of_service] [[more services]] -localdump.sh restore [Name_of_service] [file-to-restore] - - operation - one of backup|restore; optional parameter; default is backup - Name_of_service - name of database service - You get a list of all available services without parameter - Use ALL for bulk command - file - filename of db dump to restore - -Known services (see ./plugins/localdump): -couchdb -couchdb2 -ldap -mysql -pgsql -sqlite + localdump.sh [OPTIONS] <operation> <profile [more_profiles]> + +OPTIONS: + -h|--help show this help + +PARAMETERS:" + operation - one of check|backup|restore; optional parameter + backup dump all databases/ schemes of a given service + check show info only if the service is available + restore import a dump into same or new database + Without a filename it starts an interactive mode + profile - name of database profiles + You get a list of all available services without parameter + Use ALL for bulk command + file - filename of db dump to restore to origin database scheme + +EXAMPLES: + localdump.sh backup + localdump.sh backup ALL + Backup all databases of all found services + localdump.sh backup mysql + Backup all Mysql databases. + + localdump.sh restore + Start interactive restore of a database of any service. + localdump.sh restore sqlite + Start interactive restore of an sqlite database. + localdump.sh restore <file-to-restore> [<database-name>] + Restore a given dump file to the origin database scheme or + to a new/ other database with the given name. ``` If you have local Mysql daemon or Pgsql you can test it by starting ```text # dump all databases -sudo ./localdump.sh ALL +sudo ./localdump.sh backup ALL ``` ```text @@ -43,12 +61,12 @@ To dump schemes of a specific database type add the name of a known service. ```text # dump all Mysql databases -sudo ./localdump.sh mysql +sudo ./localdump.sh backup mysql ``` ## Structure in the backup folder -In the database dump folder is a subdir per service `/var/iml-backup/[service]`. +In the database dump folder is a subdir per service `/var/iml-backup/[profile]`. Below the service folder are files named like the database scheme + `__` + timestamp. @@ -59,15 +77,14 @@ will be deleted from `/var/iml-backup/[service]`. ### Backup sqlite -Keep in mind that you need to define sqlite databases in jobs/backup-dbfiles.job first. +Keep in mind that you need to create an ini file to enable sqlite backups. +See plogins/localdump/profiles/sqlite.ini.example. ```text # dump all Sqlite databases sudo ./localdump.sh sqlite ``` -This greps "^sqlite = " in jobs/backup-dbfiles.job and squentially dumps each sqlite file. - In the folder /var/iml-backup/sqlite/ it creates 2 files per database * the gzip compressed dump (filename is full path with replacing `/` by `_`)