diff --git a/docs/_index.md b/docs/_index.md new file mode 100644 index 0000000000000000000000000000000000000000..8c0fc3985a2799b239f47d1f0bf5178c4930eaad --- /dev/null +++ b/docs/_index.md @@ -0,0 +1,276 @@ +# IML deployment client + +This client is a set of bash scripts to deploy a package that was built on th IML CI server. + +This project is related to + +* CI deployment web gui <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment> +* CI package server <https://git-repo.iml.unibe.ch/iml-open-source/ci-pkg> + +## License + +GNU GPL 3.0 + +## Source + +URL: <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment-client/> + +## Installation + +On a server the client must be deployed i.e. in `/opt/deployment-client/` + +Use git clone if you feel familiar with git. Otherwise download the archive and extract it. + +## Execution plan + +If fully configured the deployment script executes the following steps in that fixed sequence: + +* loop over profiles ... and per profile ... +* set profile data by sourcing [profile]/config.js +* download Software archive +* detect if download is newer than the last one +* jump into installation dir of your application +* optional: execute pre installation tasks (1) +* optional: run cleanup (1) +* extract software archive (1) +* create config files +* optional: execute post installation tasks (1) +* optional: execute post config change tasks (1) or (2) + +(1) if a new software version was downloaded +(2) if one of the config files was created or changed + +## Set up access to package server + +This setting is for all projects on the server. It has to be done once. + +* in ./bin directory copy getfile.sh.cfg.dist to getfile.sh.cfg +* edit getfile.sh.cfg and define software endpoint and set the phase: + +```bash +IMLCI_URL=https://software.example.com +IMLCI_PKG_SECRET=put-secret-here +IMLCI_PHASE=preview +``` + +## Set up software rollout + +### Create a profile + +* Create a subdirectory in ./profiles/ for each rollout +* The example subdir gives an orientation and can be copied, i.e. `cp -r example myapp` +* create a config file named `./profiles/myapp/config.sh` (copy the config.sh.dist from example profile) + +```bash +# my install dir +installdir=/var/www/myapp + +# fileowner +# appowner="user:www-data" + +# ----- settings for CI server software package + + +IMLCI_PROJECT=id-in-ci-server + +# override global value +# IMLCI_PHASE=preview + +# cleanup after pre install tasks and bevore extracting data +# set both to 0 .. or only one of them to 1 +cleanup_preview=0 +cleanup_force=0 +``` + +Config variables: + +name | type | description +--- | --- | --- +installdir | string | target directory of your application +appowner | string | if not empty a chown -R will be applied in target directory by `chwon -R ${appowner} ${installdir}`; appowner is the parameter behind -R. It is something like "myuser." or "myuser:mygroup". For a web application it should be the user of your webservice (www-data/ apache/ nginx). The command chown requires to run the deploy script as root. +IMLCI_PROJECT | string | Project id in IML CI server +IMLCI_PHASE | string | optional: override the global IMLCI_PHASE in ./bin/getfile.sh.cfg; it is one of preview\|stage\|live +cleanup_preview | 0 or 1 | Cleanup preview - shows diff between downloaded TGZ and ${installdir}. +cleanup_force | 0 or 1 | Run cleanup: it deletes all files in target directory that aren't in the last downloaded tgz. To keep runtime data like logs or uploads you can add a file `.keep` in the directory. + +Make a testrun: `./deploy_app.sh` in application root. +It should download the software package and extract it and install it into you `${installdir}`. + +### Add hooks + +If needed you can create hook scripts. The working directory is `${installdir}` that you can use relative pathes to point to your files in the the extracted sources. + +To access other ressources you can user these variables: + +```txt +${selfdir} application root of deployment scripts +${profiledir} project config dir i.e. [path]/profiles/myapp +``` + +For hooks you can create files with pre defined names. A hook script must have executable rights. You get a hint message if it does not exist or has no x permission. A missing hook script does not result in an error. + +* profiles/myapp/tasks_preinstall.sh - do something before extracting the archive. +* profiles/myapp/tasks_config.sh - replace config files (see below) +* profiles/myapp/tasks_postinstall.sh - do postinstall actions before finishing + +#### Create configs + +The script `./bin/create_config.sh` can read config templates and create an output file. + +You need to reference the template, output file and a file for replacement data. + +```bash +# ---------------------------------------------------------------------- +# TASKS :: GENERATE CONFIGS +# ---------------------------------------------------------------------- + +# create_config.sh template_file target_file replacements (can be multiple files) +# | | | | +# v v v v +"${selfdir}/bin/create_config.sh" hooks/templates/mytemplate.erb config/target.php ${profiledir}/replace.txt +``` + +#### Pre and post install actions + +Example: A simple tasks_postinstall.sh can contain the start of a script that is delivered in the tgz archive. + +```bash +# ---------------------------------------------------------------------- +# TASKS :: POST INSTALL ACTIONS +# ---------------------------------------------------------------------- +hooks/ondeploy +``` + +## Scripts + +### deploy_app.sh + +This is the main deployment script. + +```txt +./deploy_app.sh [PROFILENAME] +``` + +If you start it without parameter it will loop over all existing profiles. +You can add an existing profile name to limit the execution to that profile only. + +### ./bin/create_config.sh + +The script is used to read a template (*.erb) to replace simple placeholders +by regex `<\%\=\ *\@replace\[\"$key\"\]\ *\%>` with a value coming from one or more +replacement data files. The output will be written to a target file. + +```txt + +===== IML DEPLOYMENT :: replace variables in erb syntax ===== + +template : +output : +replacements: replace*.txt + +ERROR: missing params. +create_config.sh TEMPLATE-FILE OUTFILE [optional: REPLACE-DATA-FILE] +``` + +A sample replacement file is profiles/example/replace.txt: + +```txt +# ---------------------------------------------------------------------- +# REPLACEMENTS +# ---------------------------------------------------------------------- +# +# SYNTAX: +# key=value +# +# - no spaces around first "=" +# - value can contain any char, no quoting +# ---------------------------------------------------------------------- + +dbhost=localhost +dbuser=fred +dbpassword=vom-jupiter +dbport=3306 + +api-key=12345678 + +# ---------------------------------------------------------------------- +``` + +The idea for several replacent files is that different groups can set +their replacement data: + +* sysadmins add connections and credentials to needed services/ database connections +* developers add api keys and other internal values +* ... + +### ./bin/getfile.sh + +With getfile.sh you can access the software archive. + +* read phase names +* read ci project ids available in a given phase +* list files of a given projects of a phase +* download a single file to a custom target file +* download ALL files to current working directory + +```txt +SYNTAX: + + getfile.sh [OPTIONS] + +OPTIONS: + + -c CFGFILE load custom config file after defaults in getfile.sh.cfg + -d enable debug infos + -e PHASE phase; overrides env variable IMLCI_PHASE + -f FILE filename to get (without path); overrides env variable IMLCI_FILE + -l ITEM list + -o OUTFILE optional output file + -p PROJECT ci project id; overrides env variable IMLCI_PROJECT + -s SECRET override secret in IMLCI_PKG_SECRET + -u URL URL of iml ci server without trailing /; overrides env variable IMLCI_URL + +VALUES: + + CFGFILE custom config file. It is useful to handle files of different + projects on a server. + PHASE is a phase of the ci server; one of preview|stage|live + FILE is a filename without path that was created by ci server. + OUTFILE Output file. It can countain a path. If none is given the filename + will be taken from FILE and stored in current directory + PROJECT project id of the ci server + SECRET secret to access project data on package server. Your given secret + must match the secret on package server to get access to any url. + ITEM type what to list; one of phases|projects|files + To list projects a phase must be set. + To list files a phase and a project must be set. + +DEFAULTS: + + You don't need to set all values by command line. Use a config to set defaults + ./getfile.sh.cfg + +EXAMPLES: + + If url, secret, project and phase are set in the config you can operate by + setting the filename to request. + + getfile.sh -f FILE + downloads FILE to the current dir. + + getfile.sh -f FILE -o my-own-filename.tgz + downloads FILE as my-own-filename.tgz + + getfile.sh -f ALL + there is a special file ALL; it fetches all filenames by executing a directory + listing and then downloads all remote files with their original name + + getfile.sh -e preview -l projects + list existing projects in phase preview + + getfile.sh -l files + list existing files of current project + + Remark: The directory listing can be turned off on the package server and + results in a 403 status. +``` diff --git a/docs/config.json b/docs/config.json new file mode 100644 index 0000000000000000000000000000000000000000..d9e12ee8f65b9276652d6bb32b27b1b4b788f9d5 --- /dev/null +++ b/docs/config.json @@ -0,0 +1,21 @@ +{ + "title": "IML deployment client", + "author": "Axel Hahn", + "tagline": "Install a software package from IML CI server.", + "ignore": { + "files": ["30_PHP-client/Plugins/Checks/_skeleton.md"], + "folders": ["99_Not_Ready"] + }, + "html": { + "auto_toc": true, + "auto_landing": false, + "date_modified": false, + "jump_buttons": true, + "edit_on_github_": "iml-it/appmonitor/tree/master/docs", + "links": { + "Git Repo": "__URL__" + }, + "theme": "daux-blue", + "search": true + } +} \ No newline at end of file diff --git a/docs/style.css b/docs/style.css new file mode 100644 index 0000000000000000000000000000000000000000..1ceb0eb44395361387353e278a5dff3fe935324c --- /dev/null +++ b/docs/style.css @@ -0,0 +1,176 @@ +/* + + patch css elements of daux.io blue theme + version 2022-04-22 + +*/ + + +/* ---------- vars ---------- */ + +:root{ + + /* background colors */ + --bg:none; + --bg-body: #fff; + --bg-navlinkactive:#f4f4f4; + --bg-navlinkactive: linear-gradient(-90deg,rgba(0,0,0,0), rgba(40,60,80,0.05) 30%); + --bg-pre:#f8f8f8; + --bg-toc: #fff; + + /* foreground colors */ + --color: #234; + --navlinkactive:#f33; + --title: #aaa; + + --link:#12a; + --toclink:rgba(40,60,80,0.8); + + --h1: rgba(40,60,80,0.8); + --h1-bottom: 1px solid rgba(40,60,80,0.1); + --h2: rgba(40,60,80,0.5); + --h3: rgba(40,60,80,0.3); + +} + +/* ---------- tags ---------- */ + +a.Brand::before { + background: rgb(255,0,51); + color: #fff; + font-family: arial; + font-weight: bold; + padding: 0.5em 0.3em; + content: 'IML'; + margin-right: 0.4em; +} + +body, *{color: var(--color);} +body{background: var(--bg-body);} + + +a{color: var(--link);} +a:hover{opacity: 0.7;} + +h1>a{ color:var(--title);} +_h1:nth-child(1){position: fixed; background: var(--bg); box-shadow: 0 0 1em #ccc; padding: 0 1em} +h1:nth-child(1)>a{ color:var(--navlinkactive); } + +.s-content h1{color: var(--h1); font-size: 200%; font-weight:bold; margin-top: 2em; border-bottom: var(--h1-bottom);} +.s-content h2{color: var(--h2); font-size: 160%; } +.s-content h3{color: var(--h3); font-size: 140%; } +.s-content h4{margin: 0; font-size: 100%; text-align: center; background-color: rgba(0,0,0,0.05);padding: 0.3em;} + +.s-content pre{ + background: var(--bg-pre); +} + +/* ---------- classes ---------- */ + +.required{color:#a42;} +.optional{color:#888;} + + +/* ----- top left */ +.Brand, +.Columns__left { + background: var(--bg); + border-right: 0px solid #e7e7e9; + color: var(--color); +} +.Brand{font-size: 200%; + background_: linear-gradient(-10deg,#fff 50%, #ddd); + background: var(--bg); +} +.Columns__right__content { + background: var(--bg); +} + +/* ----- Navi left */ + +.Nav a:hover{ + background: none; + color: var(--navlinkactive) !important; +} + +.Nav__item--active { + border-right_: 0.3em solid var(--navlinkactive); +} +.Nav__item--active > a{ + background: var(--bg-navlinkactive); + color: var(--navlinkactive); +} +.Nav .Nav .Nav__item--active a { + color: var(--navlinkactive); +} +.Nav .Nav .Nav__item a { + opacity: 1; +} +.Nav__item--open > a { + background-color: var(--bg); +} + +.Nav a[href*="__Welcome"]{ + background: url("/icons/house.png") no-repeat 10px 4px ; + padding-left: 40px; +} +.Nav a[href*="__How_does_it_work"]{ + background: url("/icons/light-bulb.png") no-repeat 10px 4px ; + padding-left: 40px; +} + + + + +/* ---------- classes ---------- */ + +/* FIX smaller fnt size in tables */ +.s-content table { + font-size: 1em; +} + + +/* TOC */ +@media(min-width:1700px){ + .TableOfContentsContainer{ + position: fixed; + right: 2em; + top: 1em; + } +} + +.TableOfContentsContainer{ + border-top-left-radius: 1em; + background-color: var(--bg-toc); + border-left: 2px solid rgba(0,0,0,0.05); + padding: 0em; +} +.TableOfContentsContainer__content { + + border: none; + font-size: 0.5em; + +} +ul.TableOfContents ul{ + list-style-type: none; + padding-left: 1em; +} +.TableOfContentsContainer a{ color:var(--toclink);} + +.TableOfContentsContainer__content > .TableOfContents > li + li { + border-top: none; +} +.TableOfContentsContainer__content > .TableOfContents > li { + border-bottom: 1px dashed #ddd; +} + +/* pager - prev .. next */ +.s-content{ + margin-bottom: 6em; +} +.Pager{ + border-top: 1px dashed #aaa; margin: 0; padding: 1em; +} +.Pager a{ + color:var(--navlinkactive); +} diff --git a/readme.md b/readme.md index 8c0fc3985a2799b239f47d1f0bf5178c4930eaad..24d4ee159dee22013619fa03c641678f17a44171 100644 --- a/readme.md +++ b/readme.md @@ -14,263 +14,3 @@ GNU GPL 3.0 ## Source URL: <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment-client/> - -## Installation - -On a server the client must be deployed i.e. in `/opt/deployment-client/` - -Use git clone if you feel familiar with git. Otherwise download the archive and extract it. - -## Execution plan - -If fully configured the deployment script executes the following steps in that fixed sequence: - -* loop over profiles ... and per profile ... -* set profile data by sourcing [profile]/config.js -* download Software archive -* detect if download is newer than the last one -* jump into installation dir of your application -* optional: execute pre installation tasks (1) -* optional: run cleanup (1) -* extract software archive (1) -* create config files -* optional: execute post installation tasks (1) -* optional: execute post config change tasks (1) or (2) - -(1) if a new software version was downloaded -(2) if one of the config files was created or changed - -## Set up access to package server - -This setting is for all projects on the server. It has to be done once. - -* in ./bin directory copy getfile.sh.cfg.dist to getfile.sh.cfg -* edit getfile.sh.cfg and define software endpoint and set the phase: - -```bash -IMLCI_URL=https://software.example.com -IMLCI_PKG_SECRET=put-secret-here -IMLCI_PHASE=preview -``` - -## Set up software rollout - -### Create a profile - -* Create a subdirectory in ./profiles/ for each rollout -* The example subdir gives an orientation and can be copied, i.e. `cp -r example myapp` -* create a config file named `./profiles/myapp/config.sh` (copy the config.sh.dist from example profile) - -```bash -# my install dir -installdir=/var/www/myapp - -# fileowner -# appowner="user:www-data" - -# ----- settings for CI server software package - - -IMLCI_PROJECT=id-in-ci-server - -# override global value -# IMLCI_PHASE=preview - -# cleanup after pre install tasks and bevore extracting data -# set both to 0 .. or only one of them to 1 -cleanup_preview=0 -cleanup_force=0 -``` - -Config variables: - -name | type | description ---- | --- | --- -installdir | string | target directory of your application -appowner | string | if not empty a chown -R will be applied in target directory by `chwon -R ${appowner} ${installdir}`; appowner is the parameter behind -R. It is something like "myuser." or "myuser:mygroup". For a web application it should be the user of your webservice (www-data/ apache/ nginx). The command chown requires to run the deploy script as root. -IMLCI_PROJECT | string | Project id in IML CI server -IMLCI_PHASE | string | optional: override the global IMLCI_PHASE in ./bin/getfile.sh.cfg; it is one of preview\|stage\|live -cleanup_preview | 0 or 1 | Cleanup preview - shows diff between downloaded TGZ and ${installdir}. -cleanup_force | 0 or 1 | Run cleanup: it deletes all files in target directory that aren't in the last downloaded tgz. To keep runtime data like logs or uploads you can add a file `.keep` in the directory. - -Make a testrun: `./deploy_app.sh` in application root. -It should download the software package and extract it and install it into you `${installdir}`. - -### Add hooks - -If needed you can create hook scripts. The working directory is `${installdir}` that you can use relative pathes to point to your files in the the extracted sources. - -To access other ressources you can user these variables: - -```txt -${selfdir} application root of deployment scripts -${profiledir} project config dir i.e. [path]/profiles/myapp -``` - -For hooks you can create files with pre defined names. A hook script must have executable rights. You get a hint message if it does not exist or has no x permission. A missing hook script does not result in an error. - -* profiles/myapp/tasks_preinstall.sh - do something before extracting the archive. -* profiles/myapp/tasks_config.sh - replace config files (see below) -* profiles/myapp/tasks_postinstall.sh - do postinstall actions before finishing - -#### Create configs - -The script `./bin/create_config.sh` can read config templates and create an output file. - -You need to reference the template, output file and a file for replacement data. - -```bash -# ---------------------------------------------------------------------- -# TASKS :: GENERATE CONFIGS -# ---------------------------------------------------------------------- - -# create_config.sh template_file target_file replacements (can be multiple files) -# | | | | -# v v v v -"${selfdir}/bin/create_config.sh" hooks/templates/mytemplate.erb config/target.php ${profiledir}/replace.txt -``` - -#### Pre and post install actions - -Example: A simple tasks_postinstall.sh can contain the start of a script that is delivered in the tgz archive. - -```bash -# ---------------------------------------------------------------------- -# TASKS :: POST INSTALL ACTIONS -# ---------------------------------------------------------------------- -hooks/ondeploy -``` - -## Scripts - -### deploy_app.sh - -This is the main deployment script. - -```txt -./deploy_app.sh [PROFILENAME] -``` - -If you start it without parameter it will loop over all existing profiles. -You can add an existing profile name to limit the execution to that profile only. - -### ./bin/create_config.sh - -The script is used to read a template (*.erb) to replace simple placeholders -by regex `<\%\=\ *\@replace\[\"$key\"\]\ *\%>` with a value coming from one or more -replacement data files. The output will be written to a target file. - -```txt - -===== IML DEPLOYMENT :: replace variables in erb syntax ===== - -template : -output : -replacements: replace*.txt - -ERROR: missing params. -create_config.sh TEMPLATE-FILE OUTFILE [optional: REPLACE-DATA-FILE] -``` - -A sample replacement file is profiles/example/replace.txt: - -```txt -# ---------------------------------------------------------------------- -# REPLACEMENTS -# ---------------------------------------------------------------------- -# -# SYNTAX: -# key=value -# -# - no spaces around first "=" -# - value can contain any char, no quoting -# ---------------------------------------------------------------------- - -dbhost=localhost -dbuser=fred -dbpassword=vom-jupiter -dbport=3306 - -api-key=12345678 - -# ---------------------------------------------------------------------- -``` - -The idea for several replacent files is that different groups can set -their replacement data: - -* sysadmins add connections and credentials to needed services/ database connections -* developers add api keys and other internal values -* ... - -### ./bin/getfile.sh - -With getfile.sh you can access the software archive. - -* read phase names -* read ci project ids available in a given phase -* list files of a given projects of a phase -* download a single file to a custom target file -* download ALL files to current working directory - -```txt -SYNTAX: - - getfile.sh [OPTIONS] - -OPTIONS: - - -c CFGFILE load custom config file after defaults in getfile.sh.cfg - -d enable debug infos - -e PHASE phase; overrides env variable IMLCI_PHASE - -f FILE filename to get (without path); overrides env variable IMLCI_FILE - -l ITEM list - -o OUTFILE optional output file - -p PROJECT ci project id; overrides env variable IMLCI_PROJECT - -s SECRET override secret in IMLCI_PKG_SECRET - -u URL URL of iml ci server without trailing /; overrides env variable IMLCI_URL - -VALUES: - - CFGFILE custom config file. It is useful to handle files of different - projects on a server. - PHASE is a phase of the ci server; one of preview|stage|live - FILE is a filename without path that was created by ci server. - OUTFILE Output file. It can countain a path. If none is given the filename - will be taken from FILE and stored in current directory - PROJECT project id of the ci server - SECRET secret to access project data on package server. Your given secret - must match the secret on package server to get access to any url. - ITEM type what to list; one of phases|projects|files - To list projects a phase must be set. - To list files a phase and a project must be set. - -DEFAULTS: - - You don't need to set all values by command line. Use a config to set defaults - ./getfile.sh.cfg - -EXAMPLES: - - If url, secret, project and phase are set in the config you can operate by - setting the filename to request. - - getfile.sh -f FILE - downloads FILE to the current dir. - - getfile.sh -f FILE -o my-own-filename.tgz - downloads FILE as my-own-filename.tgz - - getfile.sh -f ALL - there is a special file ALL; it fetches all filenames by executing a directory - listing and then downloads all remote files with their original name - - getfile.sh -e preview -l projects - list existing projects in phase preview - - getfile.sh -l files - list existing files of current project - - Remark: The directory listing can be turned off on the package server and - results in a 403 status. -```