add first doc pages

e5443194 · hahn · 2c56ffb9 · e5443194 · e5443194 · e5443194
Commit e5443194 authored 3 years ago by hahn
--- a/docs/10_Server/Processes/10_Build.md
+++ b/docs/10_Server/Processes/10_Build.md
+# Build #
+
+A build process can be started ...
+
+* by `Build`button on overview page
+* by `Build`button in application view
+* by API call
+
+Among its steps are some builtin, some depend on the project settings and some can be influenced by the developers.
+
+A build is denied if a project has no activated phase the project settings.
+
+## Overview ##
+
+![Build process](../../../docs/images/processes-build.png "Build process")
+
+## Steps ##
+
+### Create working directory ###
+
+Outdated kept builds will be cleaned up.
+
+Below `/var/imldeployment/build/` a subdirectory with the id of the project will be created. In that one a uniq directory will be created by using a timestamp.
+
+### Get sources ###
+
+The `git clone` fetches the sources from the Git repository
+
+Remark: at the moment ony Git is suported as version control system. An interface in the classes subdir describes the required method to be implemented. An additional VCS will be available in the project settings if it was added.
+
+### Project specific actions ###
+
+An `chmod 755 /hooks/on*` makes hook scripts executable.
+
+These steps will be executed if they exist:
+
+* `hooks/onbuild-postclone` - actions after cloning the sources
+* Sync other default files that are not in the repository - it was needed for rarely special projects.
+* `hooks/onbuild` - actions before runinng the build or compression
+
+Recommendation:
+
+Use a script named `hooks/onbuild` and put your project specisif actions into it.
+Read the next chapters to use given environment and scripts.
+
+#### Custom vars ####
+
+A file named `ci-custom-vars` will be created containing the given data in the project config. Its idea is to be sourced by hook script to set variables in the current shell.
+
+Example:
+
+If it contains
+
+```shell
+export myVar="hello world"
+```
+
+it can be sourced with this snippet:
+
+```shell
+cd `dirname $0`
+cd ..
+. ci-custom-vars || exit 1
+echo myVar=$myVar
+```
+
+#### Builtin environment ####
+
+* $GIT_SSH - full path to git ssh wrapper (CI-Root/shellscripts/gitsshwrapper.sh)
+* $DIR_SSH_KEYS - full path to ssh keys (/var/imldeployment/data/sshkeys)
+* $DIR_APPROOT - full path of the current build directory (z.B. /var/imldeployment/build/ci-webgui/ci-webgui_20171211-102707)
+* $RVMSCRIPT - for Rails projects: path to the RVM script. With it you can set a custom Ruby version
+* $NVMINIT - for NodeJs projects - to install a custom node version
+
+Snippets:
+
+(1)
+
+Set a Ruby version:
+Rvm must be installed on the server where the ci server runs.
+
+```shell
+. $RVMSCRIPT || exit 1
+rvm use 2.2.3
+```
+
+(2)
+
+Set a custom Nodejs version:
+The NVM init script is part of the ci server.
+
+```shell
+. $RVMSCRIPT || exit 1
+rvm use 2.2.3
+. $NVMINIT || exit 1
+nvm install [Version]
+```
+
+Important: at the end of the hook script uninstall it by using `nvmremove`.
+
+ Ressources:
+
+* CI-Git-Repo .. nvm_init.sh: <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/-/blob/master/shellscripts/nvm_init.sh>
+* NVM: <https://github.com/nvm-sh/>
+
+### Remove vcs data ###
+
+The version control data (`.git` directory in build root) will be removed.
+
+### Build or compress ###
+
+(1)
+A metafile `[Projecd_ID].json` will be created in the build root. It will be used to identify the package or installation. It contains date, branch and commit message.
+
+(2)
+WIP
+
+All available built plugins have the name of the subdirs below `public_html/deployment/plugins/build/`.
+
+A loop over available plugins will execute the activated build plugins to initialize one or builds.
+
+The output dir is `/var/imldeployment/packages/_files/[Project_ID]/`. Here are subdirectories with the timestamp when the build was started, e.g. `./20220705_115145/`
+
+(3)
+If there are files in hooks/templates/ the will be copied into the package output directory. They can be used by automation tools like puppet to generate configuration files for different targets and different phases.
+
+### Remove build dir ###
+
+If all actions were successful the buld directory will be deleted.
+
+### Add in queue of 1st phase ###
+
+A successful build triggers the queuing to the first active phase of this project.
+
+A phase can define deploy times to define, when a rollout is allowed. In the special case that the deploy time has no limit it will be installed instantly. Otherwise you will see the package in the queue column of a phase.
+
+## If a build fails ##
+
+The working directory will be kept. In the project config the number of kept projects is set (default: 3). Additionally old failed builds (older 7 days) will be deleted by a cronjob.
+
+If you open the project in the web ui you find the kept folders in the tab "Build errors".
+
+A sysadmin with access to the ci server can switch to the build directory and repeat the execution of a hook script.
--- a/docs/_index.md
+++ b/docs/_index.md
@@ -20,7 +20,6 @@ The archives can be synched to multiple deployment targets e.g. puppet master or

 ## Features ##

-* API to start a build from somewhere, e.g. from a devops workplace or Gitlab server
 * checkout from git via SSH with multiple ssh keys (can be extended with a plugin)
 * build has hooks to customize build process
 * In our institute it builds projects written in
@@ -31,6 +30,7 @@ The archives can be synched to multiple deployment targets e.g. puppet master or
 * trigger rollout via ssh command or AWX API call (can be extended with a plugin)
 * receives install status 
 * sends messages (email, Slack)
+* API to start a build from somewhere, e.g. from a devops workplace or Gitlab server

 ## file structure ##

@@ -38,18 +38,74 @@ The archives can be synched to multiple deployment targets e.g. puppet master or
 * data dir - configuration, database, built archives
 * temp area - checked out projects to read comit messages

+### Approot and website ###
+
+Default: /var/www/[YOUR-DOMAIN]/
+
+```txt
+.
+├── api
+├── appmonitor
+│   ├── classes
+│   └── plugins
+│       └── checks
+├── ~cache
+├── deployment
+│   ├── classes
+│   │   └── tests
+│   ├── images
+│   ├── js
+│   ├── pages
+│   └── plugins
+│       ├── build
+│       │   └── tgz
+│       └── rollout
+│           ├── awx
+│           ├── default
+│           └── ssh
+├── valuestore
+│   ├── classes
+│   ├── data
+│   └── tests
+├── vendor
+│   ├── bootstrap3
+│   ├── font-awesome
+│   ├── font-awesome-4.7.0
+│   │   ├── css
+│   │   └── fonts
+│   ├── jquery
+│   │   ├── 3.3.1
+│   │   └── 3.4.1
+│   ├── medoo
+│   │   └── src
+│   ├── shooker
+│   ├── spyc
+│   └── vis
+│       └── 4.21.0
+│           └── img
+│               └── network
+├── versions
+│   ├── classes
+│   └── data
+└── webservice
+```
+
 ### Data ###

 By default: /var/imldeployment

 ```txt
 imldeployment/
-├── build
+├── build          << build directories
 ├── data
 │   ├── database
 │   ├── projects
 │   └── sshkeys
 ├── defaults
-└── packages
+└── packages       << output data of buils
    └── _files
 ```
+
+### Temp ###
+
+By default: /var/tmp/imldeployment
--- a/docs/config.json
+++ b/docs/config.json
 {
-    "title": "IML Appmonitor",
+    "title": "IML CISERVER",
    "author": "Axel Hahn",
-    "tagline": "Application monitoring with low requirements.",
+    "tagline": "Build and rollout projects",

    "html": {
        "auto_toc": true,
@@ -9,12 +9,12 @@
        "date_modified": false,
        "jump_buttons": true,
        "edit_on_github_": "iml-it/appmonitor/tree/master/docs",
-        "edit_on_": {
+        "edit_on": {
            "name": "Gitlab",
-            "basepath": "__"
+            "basepath": "https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/tree/master/docs"
        },
        "links": {
-            "GitHub Repo": "https://github.com/iml-it/appmonitor"
+            "GitHub Repo": "https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/"
        },
        "theme": "daux-blue",
        "search": true

--- a/docs/images-sources/processes-build.drawio
+++ b/docs/images-sources/processes-build.drawio
+<mxfile host="Electron" modified="2022-07-21T12:42:31.740Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/19.0.3 Chrome/100.0.4896.160 Electron/18.3.5 Safari/537.36" etag="S7cbsLsyLaclOyHWU7L3" version="19.0.3" type="device"><diagram id="u_O4M5wh5bko4jYocOWh" name="Page-1">5Vxtc6M2EP41nmk/1APi1R9jX+4y07RNm5n27lNGBhmrwYgKkcT99ZVAYIMIJjUY25dL5pAQWHqe3dXuSvLEWGzevlAYr38hPgonQPPfJsanCQC6CcBE/Gr+Nq+ZaVZeEVDsy0a7ikf8L5KVmqxNsY+SSkNGSMhwXK30SBQhj1XqIKXktdpsRcLqp8YwQErFowdDtfYv7LN1XusCZ1d/h3CwLj5Zt2f5nQ0sGsuRJGvok9e9KuN2YiwoISy/2rwtUCjAK3DJn/v8zt2yYxRFrMsDW39tpm/RywO7Cxa/LX+2n+7Nnww3f80LDFM5Ytlbti0goCSNfCTeok+M+esaM/QYQ0/cfeWk87o124Tytg+Tddk2YZQ8owUJCc1eZcxsx4A2v7PCYVjURyRCZeMCZCBqnhHz1vJdsp+IMvT2LgJ6iSsXSEQ2iNEtbyIfME1JhZRFoMny645Z05L8rfdZncmGUEpTUL57Bzi/kJh/AH9HQRv5XP5kkVC2JgGJYHi7q53v+NB4adfmnpBYgvU3YmwrlQmmjFQ54nDR7Vf5fFb4JgpTqyh+etu/+WkrSw18iN62s8EHR1LqoRYQCr2GNECspZ3dzC5FIWT4pdqP3pnSFUX5A8VEoa+CUany2r5maFU2qkoilSGESxQ+kAQzTCJe7XFQEb8/FwqAuXm6rzVYEsbIZq/BTYgDcYMJoZiTlIU44p9SWEnRCSiblC/n/Y3FODZvgbDmU7JaYQ9NfcjgEiYoKa+eNjjCT6CmyFzBgeaAuSvrCzFOSCig7UWFHbeqwqalqLCtqRpsD6XA+igaPJQmgo6aqJtjqiJQVPGR95hNgA03QtijZSL+g7zNMsV8wj9qRntfo4MQJom87kG2y+lIyrarzk5lk5PItqni/Hininvk3whHSxgSAQj2qvhVhT0RTBXtpbGr2RCk+xZyWt2HTr5CP3OT0VUljtQI+egDwbyLpUxYoOaygBrZ+QDkU/veX+1Ftl19Ud2jycenvCeTm3I4R5hJ1a38Dh0du6Mszca0rrai9aKz3JRSGHlrHovVzCwvh3ww8yXlV4G4gpGfjTJIWiyvdtjy/g+70HPEYM0Ouxs6OKVNVuO1DUyE99Yn0itL/BP1JGJ79Zpm8L8mBuzsZwAGnCIeqxvAQwwYQzEwUxjINeNJ75WDEaRdwdoeG+siC6SC/ev3IvBGQ5LixCSooS+37U9fv303FJhjU2C5re6T9KOP9pfaoT8vd3tUH6no5p5KfMmcpHyAF+/3gHqmtCEWPa3fY4ycKd3FDN8m+yHDiTOlZlf1MEZVDzVzsKAIMiQ+TEgRoc84CuqRhI8p8hiRHehLfXyI3JXXOIF4LlquTqA++ujqA2YXpT4NSaLBUp5dVx8AOFKlskf5qOB2r0EsMi6Jwm8PKZfCGbkQyoeit2tKGxxrMZvzbqCWd3M7Jt5OJycFQvvGOkQwSmM1s0PRhrwIKz4NMLs2J8ce3UobagrhkheTuvryYNTVJEu1k8cHUrXYdoVsr9EF8Z3Z8p0QawwmTGdMJoAaVT2EaYBxNDFuVGNUupTcxjzDAP2Q/NirSerC2sAmyXGc6dgZZ8O+KqPUNYIC1mW5e0CNuTg6+Wwt4i0eXPWqHWcQVrlnoB3AUmD/PUWpQJ2LNND0RDiD0MtEhg92DRN0bTzMzoAHQ12WaYH56I2M5QwwzkbG2q4AR1c9V6MJfl23BsK/EIk9/P9cPAr8Y+ThFfakFpDo4pOjZZxQiH/T8sCsYRtpfe9Gf+CrKzRemoj9gHxclPwttvwNzgTnwfXNJiZcsDQGWahRmHBGZ0INpjMXtmesR3BN61jrWtPC5GnBVuMFsd+P8XhhIBE//VyriHiDqT8x6iOnkidnsZXLlOtRh0Nqt5nf04TURTf3k3tr5D1PhHio2b1/UkzRBhUhygWbK9PUCp+0cJKM8b1UU92+Nc93KGscEXGKaRNTlFw8+tZsdoboW+rkfI8CJHYr1uDmI2eTw6c0Gpx/5UxF/TzGBvt+ZhKbSKztlS5Y6WmDeX0ucZv2dzXNJcPta2lf1b+ifS2zjtPF3tHAMRbu1Sj6jpDnLP3aMGHEJGFeKEi68ECintEwRl8KstqzrtejGQWqhzXj2BMWx+X51PzGgsTi8320gmnIVO3gGKYeS+nVaYc5unYANeHRaqcoGsSzOgMurNG5sNQ5IzM/3KV9EecS8uQ3ZEh4ufI8nha/kxQ5Q8frPRPZA5mmUSXTbshfGU1H/gZzyGzVzLVozEdPTzYdCm5RHWUOaqC9kqbvg5LaAWOzKbnrNFCi64NxooYtNNlG3scdhAoZxyRh9KnmGtVEjH0gEZOVHhDFHBShn6261U59N8vbo49id94jOKr3bquzYumcaBxAsS9dizObXK4K1ESoIiHZYjWity9ZQuiqvtLAI5GHYpZMxRfCZA1rAuZ8nt+YheDVbdaAU4Jl1OxPQ4xuNZgf8+MLe7y4+/aZfBvC7jt8jNv/AA==</diagram></mxfile>
\ No newline at end of file
--- a/docs/images/processes-build.png
+++ b/docs/images/processes-build.png