diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..c2b1387622398881edd0e218c1d40f5604b477a1 --- /dev/null +++ b/LICENSE @@ -0,0 +1,166 @@ +GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the GNU +General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this license + document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this license + document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of this + License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. + diff --git a/config/config_custom.php.dist b/config/config_custom.php.dist new file mode 100644 index 0000000000000000000000000000000000000000..a22106245a9f4a0aece30807a312b575b6881a46 --- /dev/null +++ b/config/config_custom.php.dist @@ -0,0 +1,133 @@ +<?php +/* + * CUSTOM SETTINGS + * + * Settings here override defaults from config_defaults.php + * + */ +return [ + + 'lang' => 'en-en', // for available languages see ./config/lang/*.json + + 'auth' => [ + // force using a given user ... for development only + 'forceuser' => 'admin', + + // use a real login + 'ldap' => [ + 'server' => 'ldaps://ldap.example.com', + 'port' => 636, + 'DnLdapUser' => 'cn=lookupuser,dc=department,dc=example.com', + 'PwLdapUser' => 'lookupuser_password_here', + 'DnUserNode' => 'ou=People,dc=department,dc=example.com', + 'DnAppNode' => 'cn=CI Web-GUI Users,ou=Application Access,dc=department,dc=example.com', + 'debugLevel' => 0, + ] + ], + + 'banner' => '', + + // ---------------------------------------------------------------------- + + 'phases' => [ + "preview" => [], + "stage" => [], + "live" => [ + // prevent immediate installation after build or accept + "deploytimes" => ['/(Mon|Tue|Wed|Thu)\ 14\:/'], + ], + ], + 'showdebug' => [ + 'ip'=> ['127.0.0.1'], + ], + + 'projectgroups' => [ + 'teamA'=>'Team A: Services', + 'teamB'=>'Team B: Website', + ], + + // ---------------------------------------------------------------------- + // build settings + // ---------------------------------------------------------------------- + 'versionsToKeep' => 10, // for cleanup: keep n unused versions + 'builtsToKeep' => 3, + 'build' => [ + 'env' => 'export RVMSCRIPT="/usr/local/rvm/scripts/rvm";', + 'hooks' => [ + 'build-postclone' => 'hooks/onbuild-postclone', + 'build-precompress' => 'hooks/onbuild', + ], + ], + + // ---------------------------------------------------------------------- + // rsync of archives + // ---------------------------------------------------------------------- + 'mirrorPackages' => [ + /* + + // (1) + // sync to a puppet master puppet to extract archive and generate templates + 'puppet' => [ + 'type' => 'rsync', + 'runas' => '', // www-data, // nur fuer commandline + 'target' => 'copy-deployment@puppetmaster.example.com:/share/ciserver', + ], + + // (2) + // sync to a software package server like https://os-docs.iml.unibe.ch/ci-pkg/ + 'package-server' => [ + 'type' => 'rsync', + 'runas' => '', // www-data, // nur fuer commandline + 'target' => 'copy-deployment@software.example.com:/var/www/data-ciserver', + ], + */ + ], + + // ---------------------------------------------------------------------- + // plugins + // existing subkeys = enabled plugins + // ---------------------------------------------------------------------- + 'plugins'=>[ + + 'rollout'=>[ + 'default'=>[], + 'ssh'=>[ + 'user'=>'imldeployment', + 'privatekey'=>'', + 'addkeycommand'=>'/usr/bin/ssh-keygen -R %s; /usr/bin/ssh-keyscan -t rsa %s >> /home/www-data/.ssh/known_hosts', + 'testcommand'=>'sudo puppet --version', + 'command'=>'/usr/local/bin/puppetrun.sh', + ], + 'awx'=>[ + 'url'=>'https://awx.sys.iml.unibe.ch/api/v2', // no ending "/" + 'user'=>'api-ci', + 'password'=>'awRSbdB2rkViaBXBKOvtr11DEoZJSqHceih1hEE4awrjIO1wuArKu85WmetsRp63', + 'jobtemplate'=>'36', + 'tags'=>'rollout', + // 'ignore-ssl-error'=>false, + ], + ], + ], + + // ---------------------------------------------------------------------- + // notifications to messengers ... + // ---------------------------------------------------------------------- + + // notifications to messengers ... + 'messenger'=>[ + 'slack'=>[ + 'presets'=>[ + 'https://hooks.slack.com/services/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'=>[ + 'label'=>'#medsurf-heartbeat', + 'user'=>'[CI-WebGUI]', + ], + 'https://hooks.slack.com/services/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'=>[ + 'label'=>'#msrd-analyzer-hrtbeat', + 'user'=>'[CI-WebGUI]', + ], + ], + ], + ], + // ---------------------------------------------------------------------- + +]; \ No newline at end of file diff --git a/config/config_defaults.php b/config/config_defaults.php new file mode 100644 index 0000000000000000000000000000000000000000..0cb6c90ef8f35e80fdb14959f0cbdedd9ec3d27a --- /dev/null +++ b/config/config_defaults.php @@ -0,0 +1,170 @@ +<?php +/* + * DEFAULT CONFIG SETTINGS + * + * Do not change this file. For custom settings use the config_custom.php + * and override wanted keys. + * + */ +return [ + + // ---------------------------------------------------------------------- + + 'workDir' => '/var/imldeployment', + 'tmpDir' => '/var/tmp/imldeployment', + + // ---------------------------------------------------------------------- + + 'phases' => [ + "preview" => [ + 'css' => [ + 'bgdark' => 'background:#393E50; color:#f8f8f8;', + 'bglight' => 'background:#eee; color:#333; background:rgba(210,210,210,0.3); ', + 'bgbutton' => 'background:#393E50; color:#fcfcfc; border: 1px solid rgba(0,0,0,0.15);', + ], + ], + "stage" => [ + 'css' => [ + 'bgdark' => 'background:#3F88C5; color:#f8f8f8;', + 'bglight' => 'background:#f0f4f8; color:#333; background:rgba(200,210,220,0.3); ', + 'bgbutton' => 'background:#3F88C5; color:#fcfcfc; border: 1px solid rgba(0,0,0,0.15);', + ], + ], + "live" => [ + 'css' => [ + 'bgdark' => 'background:#44BBA4; color:#f8f8f8;', + 'bglight' => 'background:#f4f8f0; color:#333; background:rgba(210,220,200,0.3); ', + 'bgbutton' => 'background:#44BBA4; color:#fcfcfc; border: 1px solid rgba(0,0,0,0.15);', + ], + // prevent immediate installation after build or accept + "deploytimes" => ['/(Mon|Tue|Wed|Thu)\ 14\:/'], + ], + ], + + 'auth' => [ + // force using a given user ... for development only + 'forceuser' => 'admin', + + // use a real login + 'ldap' => [ + 'server' => 'ldaps://ldap.example.com', + 'port' => 636, + 'DnLdapUser' => 'cn=lookupuser,dc=department,dc=example.com', + 'PwLdapUser' => 'lookupuser_password_here', + 'DnUserNode' => 'ou=People,dc=department,dc=example.com', + 'DnAppNode' => 'cn=CI Web-GUI Users,ou=Application Access,dc=department,dc=example.com', + 'debugLevel' => 0, + ] + ], + + 'lang' => 'en-en', // for available languages see ./config/lang/*.json + + 'showdebug' => [ + 'ip'=> ['127.0.0.1'], + ], + 'projectgroups' => [], + 'banner' => '', + + // ---------------------------------------------------------------------- + // build settings + // ---------------------------------------------------------------------- + 'versionsToKeep' => 10, // for cleanup: keep n unused versions + 'builtsToKeep' => 3, + 'build' => [ + 'env' => 'export RVMSCRIPT="/usr/local/rvm/scripts/rvm";', + 'hooks' => [ + 'build-postclone' => 'hooks/onbuild-postclone', + 'build-precompress' => 'hooks/onbuild', + ], + ], + + // ---------------------------------------------------------------------- + // sync of archives + // ---------------------------------------------------------------------- + 'mirrorPackages' => [ + /* + + // (1) + // sync to a puppet master puppet to extract archive and generate templates + 'puppet' => [ + 'type' => 'rsync', + 'runas' => '', // www-data, // nur fuer commandline + 'target' => 'copy-deployment@puppetmaster.example.com:/share/ciserver', + ], + + // (2) + // sync to a software package server like https://os-docs.iml.unibe.ch/ci-pkg/ + 'package-server' => [ + 'type' => 'rsync', + 'runas' => '', // apache httpd user is default, e.g. www-data + 'target' => 'copy-deployment@software.example.com:/var/www/data-ciserver', + ], + */ + ], + + // ---------------------------------------------------------------------- + // plugins + // existing subkeys = enabled plugins + // ---------------------------------------------------------------------- + 'plugins'=>[ + 'build'=>[ + 'tgz'=>[], + 'zip'=>[], + ], + 'rollout'=>[ + 'default'=>[], + /* + 'ssh'=>[ + 'user'=>'deployment', + 'privatekey'=>'', + 'addkeycommand'=>'/usr/bin/ssh-keygen -R %s; /usr/bin/ssh-keyscan -t rsa %s >> /home/www-data/.ssh/known_hosts', + 'testcommand'=>'sudo puppet --version', + 'command'=>'/usr/local/bin/puppetrun.sh', + ], + 'awx'=>[ + 'url'=>'https://awx.example.com/api/v2', // no ending "/" + 'user'=>'ciserver', + 'password'=>'ciserver', + 'jobtemplate'=>'36', + 'tags'=>'rollout', + ], + */ + ], + ], + + // ---------------------------------------------------------------------- + // notifications to messengers ... + // ---------------------------------------------------------------------- + 'messenger'=>[ + 'slack'=>[ + 'presets'=>[], + ], + 'email'=>[ + 'from'=>'noreply@ciserver.example.com', + ] + ], + + // ---------------------------------------------------------------------- + // TODO: functionality to be removed?! + // ---------------------------------------------------------------------- + + 'foreman__' => [ + 'api'=>'https://foreman.example.com/', // with ending "/" + 'user'=>'ci-server', + 'password'=>'ciserver_password_here', + 'ignore-ssl-error'=>true, + // 'varname-replace'=>'ci-replacement', + ], + + // where to store project data + 'projects' => [ + 'json' => [ + 'active' => true, + ], + 'ldap' => [ + 'active' => false, + ], + ], + // ---------------------------------------------------------------------- + +]; \ No newline at end of file diff --git a/config/inc_projects_config.php b/config/inc_projects_config.php new file mode 100644 index 0000000000000000000000000000000000000000..0345ab7affa26af284cbbff71d9e6cc83aca0136 --- /dev/null +++ b/config/inc_projects_config.php @@ -0,0 +1,23 @@ +<?php + +$aConfig = include('config_defaults.php'); +if (file_exists(__DIR__.'/config_custom.php')){ + $aConfig = array_replace_recursive( + $aConfig, + include('config_custom.php') + ); +} + +// ---------------------------------------------------------------------- +// generate some vars +// ---------------------------------------------------------------------- + +$aConfig = array_merge($aConfig, [ + 'appRootDir' => dirname(__DIR__), + 'configDir' => __DIR__, + 'dataDir' => $aConfig['workDir'] . '/data', // to write data: ssh keys, projects, database + 'buildDir' => $aConfig['workDir'] . '/build', + 'buildDefaultsDir' => $aConfig['workDir'] . '/defaults', + 'packageDir' => $aConfig['workDir'] . '/packages', + 'archiveDir' => $aConfig['workDir'] . '/packages/_files', +]); diff --git a/config/inc_roles.php b/config/inc_roles.php index 0db2b60f59aada087f0410e4b0911cda63fc0c10..018c7205c96d4777d145db93a0f91f410af3a9fd 100644 --- a/config/inc_roles.php +++ b/config/inc_roles.php @@ -7,11 +7,11 @@ * */ -return array( - "all" => array( +return [ + "all" => [ "page_login" - ), - "authenticated" => array( + ], + "authenticated" => [ "page_overview", "page_accept", @@ -34,10 +34,10 @@ return array( "project-action-overview", "project-action-phase", "project-action-setup", - ), + ], - "admin" => array( + "admin" => [ "page_accept", "page_build", @@ -46,6 +46,7 @@ return array( "page_deploy", "page_doc", "page_htmltest", + "page_installer", "page_phase", "page_setup", "page_checkssh", @@ -63,16 +64,16 @@ return array( "project-action-phase", "project-action-setup", "project-action-setup-edit-replacements", - ), + ], // ----- wenn es mal eine feinere Granulierung braucht, muss man eine // User-Admin programmieren /* - "authenticated_" => array( + "authenticated_" => [ "page_overview", - ), - "developer_" => array( + ], + "developer_" => [ "page_build", "page_cleanup", "page_setup", @@ -85,15 +86,15 @@ return array( "project-action-overview", "project-action-phase", "project-action-setup", - ), - "projectmanager_" => array( + ], + "projectmanager_" => [ "project-action-default", "project-action-accept-preview", "project-action-accept-stage", // "project-action-deploy", "project-action-overview", "project-action-phase", - ), + ], */ -); +]; diff --git a/config/inc_user2roles.php.dist b/config/inc_user2roles.php.dist new file mode 100644 index 0000000000000000000000000000000000000000..133af67ae858e0b804a997d439b3f97112abf068 --- /dev/null +++ b/config/inc_user2roles.php.dist @@ -0,0 +1,22 @@ +<?php +/* + * IML DEPLOYMENT GUI + * LIST OF ROLES AND ASSIGNED USERS + * + * A user can be assigned to multiple groups + * + * see permissions per role in inc_roles.php + * + * Remarks: + * - Non authenticated users are member in "all" + * - Every user logged in is member of "authenticated" + */ +return [ + "developer" => [], + "projectmanager" => [], + "admin" => [ + 'admin', + 'anotheradminuser', + ], + +]; diff --git a/config/lang/de.json b/config/lang/de-de.json similarity index 97% rename from config/lang/de.json rename to config/lang/de-de.json index 6776db8fe94659334796e08c491422c430565b09..6404074b4f20bc3e4e396d8236d4d36000ce8f0b 100644 --- a/config/lang/de.json +++ b/config/lang/de-de.json @@ -309,8 +309,10 @@ "project": "Projekt", "project-home": "Projekt Startseite", "projectdescription": "Kurzbeschreibung", + "projectgroup": "Projektgruppe", "projectname": "Projektname", "projectmanager": "Projektleiter", + "project-setup-incomplete": "Ooops: Die Projekt-Einstellungen sind noch nicht abgeschlossen. Gehen Sie in die Projekt-Einstellungen.", "queue": "Queue", "queue-hint-overview": "neuestes Paket<br>in der ersten<br>Phase %s<br>installieren", "raw-data": "Raw data", @@ -346,7 +348,7 @@ "yes": "ja", "onhold": "Queue", - "ready2install" : "Puppet", + "ready2install" : "Bereit", "deployed" : "Installiert", diff --git a/config/lang/en.json b/config/lang/en-en.json similarity index 96% rename from config/lang/en.json rename to config/lang/en-en.json index fb01122710bc61359e89894d7b8bbcdbcbfe5213..941b70d5484bc4ab9d1b64ace04fda253e3c69cf 100644 --- a/config/lang/en.json +++ b/config/lang/en-en.json @@ -65,7 +65,7 @@ "class-project-deploy-label-checks": "Checks", "class-project-deploy-label-activate-queued-version": "Activate the queued version", - "class-project-deploy-label-synch-packages": "Synchronize package to the puppet master", + "class-project-deploy-label-synch-packages": "Synchronize package to targets.", "class-project-deploy-label-install-on-target": "Installation on the target system", "class-project-error-_getArchiveDir-requires-id": "invalid call: The method _getArchiveDir requires a version number.", @@ -149,7 +149,7 @@ "class-project-warning-cannot-delete-build-dir": "WARNING: The Build directory %s could not be deleted.", "class-project-warning-phase-not-active": "The phase %s is not active.", - "class-user-error-deny-no-role": "ERRROR: Your User has not enough permissions.", + "class-user-error-deny-no-role": "ERROR: Your User has not enough permissions.", "class-user-error-login-required": "ERROR: You need to login first.", "page-accept-error-cannot-accept-phase": "The phase [%s] cannot be accepted.", @@ -235,7 +235,7 @@ "build-failes-none": "No failed builds were recorded yet.", "change": "Change", "commitmessage": "Commit message", - "contact": "contact", + "contact": "Contact", "cleanup": "Cleanup", "creating-directory": "Creating directory %s", "creating-file": "Creating file %s", @@ -309,10 +309,12 @@ "progress-inprogress": "in progress", "progress-hasqueue": "waiting for installation", "project": "Project", - "project-home": "Projekt home", + "project-home": "Project home", "projectdescription": "Short description", + "projectgroup": "Project group", "projectname": "Project name", - "projectmanager": "Projekt manager", + "projectmanager": "Procekt manager", + "project-setup-incomplete": "Ooops: The project settings are not finished. Got to the Setup.", "queue": "Queue", "queue-hint-overview": "Install newest<br>package in the first<br>phase %s", "raw-data": "Raw data", @@ -323,7 +325,7 @@ "repositoryinfos": "Sourcecode Repository", "repository-access-browser": "Browser access to sources", "repository-auth": "Authentication/ filename of ssh private key", - "repository-has-public-dir": "Is it a web projekt with subdirectory <em>public</em> or <em>public_html</em>?", + "repository-has-public-dir": "Is it a web project with subdirectory <em>public</em> or <em>public_html</em>?", "repository-url": "URL to repository", "repository-urlwebgui": "URL zur Web GUI des Repositorys", "revision": "Revision", @@ -348,8 +350,8 @@ "yes": "yes", "onhold": "Queue", - "ready2install" : "Puppet", - "deployed" : "deployed", + "ready2install" : "Ready", + "deployed" : "Deployed", "lastentry": "This is just a Dummy entry without comma at the end of line" diff --git a/data/imldeployment/.htkeep b/data/imldeployment/.htkeep new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/data/imldeployment/build/readme.md b/data/imldeployment/build/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..b627860bd4ceaec43da82f0d3356e237f755106d --- /dev/null +++ b/data/imldeployment/build/readme.md @@ -0,0 +1,3 @@ +# data/imldeployment/build # + +place where projects will be cloned and builds will be started diff --git a/data/imldeployment/data/database/readme.md b/data/imldeployment/data/database/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..ae2bdcadd1e1c8a63e27bf40884ef7196fb1c157 --- /dev/null +++ b/data/imldeployment/data/database/readme.md @@ -0,0 +1,3 @@ +# data/imldeployment/data/database # + +sqlite database for logs \ No newline at end of file diff --git a/data/imldeployment/data/projects/readme.md b/data/imldeployment/data/projects/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..35ef415f86869b4de538a9bcafdca3c025059034 --- /dev/null +++ b/data/imldeployment/data/projects/readme.md @@ -0,0 +1,3 @@ +# data/imldeployment/data/projects # + +configuration files for all projects diff --git a/data/imldeployment/data/sshkeys/readme.md b/data/imldeployment/data/sshkeys/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..038137c14d899113df20591024fbccdaa7e107b7 --- /dev/null +++ b/data/imldeployment/data/sshkeys/readme.md @@ -0,0 +1,3 @@ +# data/imldeployment/data/sshkeys # + +ssh private key files to connect git repository servers with ssh. diff --git a/data/imldeployment/defaults/readme.md b/data/imldeployment/defaults/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..33d41a7f41638ca4e38d622f3759014e90f2dff8 --- /dev/null +++ b/data/imldeployment/defaults/readme.md @@ -0,0 +1,11 @@ +# data/imldeployment/defaults # + +A place to share data for builds. + +Normally a project will be cloned and starts the build. +You can sync a bunch of files and dirs into the build dir after cloning. + +## Howto ## + +- Create a subdir with the id of the project +- put your needed files and folders into it diff --git a/data/imldeployment/packages/readme.md b/data/imldeployment/packages/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..efddb541fee58993a80953704a971c7489b616d3 --- /dev/null +++ b/data/imldeployment/packages/readme.md @@ -0,0 +1,5 @@ +# data/imldeployment/packages # + +subdir _files is the place of created archives. + +Additional dirs exist for each defined phase (e.g. preview, stage, live) diff --git a/docker/.env b/docker/.env new file mode 100644 index 0000000000000000000000000000000000000000..40b752e095652f103dd2b49dc0611c85d9e60c69 --- /dev/null +++ b/docker/.env @@ -0,0 +1,16 @@ +# ====================================================================== +# +# GENERATED BY docker/init.sh - template: ./templates/dot_env - e2cde05722688ff85d3a93e9cd55787e +# values to be used in docker-composer.yml +# +# ====================================================================== + +# ----- application +APP_NAME=imlcinode + +# uid of www-data in the docker container +DOCKER_USER_UID=33 + +APP_PORT=8002 +WEBROOT=/var/www/imlcinode/public_html + diff --git a/docker/containers/web-server/Dockerfile b/docker/containers/web-server/Dockerfile new file mode 100644 index 0000000000000000000000000000000000000000..9741db3ef038d3e8fe395406f3565100abb74d6b --- /dev/null +++ b/docker/containers/web-server/Dockerfile @@ -0,0 +1,14 @@ +# +# GENERATED BY docker/init.sh - template: ./templates/web-server-Dockerfile - 42dce773c83597a7d05af398bdd66d15 +# +FROM php:8.1-apache + +# install packages +RUN apt-get update && apt-get install -y git unzip zip rsync + +# enable apache modules +RUN a2enmod rewrite + +# install php packages +COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/ +RUN install-php-extensions curl mbstring ldap intl xml diff --git a/docker/containers/web-server/apache/sites-enabled/vhost_app.conf b/docker/containers/web-server/apache/sites-enabled/vhost_app.conf new file mode 100644 index 0000000000000000000000000000000000000000..c223a7617906a709c511b1192b28d61fb0437c6c --- /dev/null +++ b/docker/containers/web-server/apache/sites-enabled/vhost_app.conf @@ -0,0 +1,36 @@ +# +# GENERATED BY docker/init.sh - template: ./templates/vhost_app.conf - 9a9cf79de5a3584c0cef6cb79c339c25 +# + + +# load modules +# LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so + + +<VirtualHost *:80> + DocumentRoot /var/www/imlcinode/public_html + + <Directory "/var/www/imlcinode/public_html/deployment/"> + + # AuthUserFile /var/www/imlcinode/public_html/deployment/.htusers + # AuthName "IML DEPLOYMENT GUI" + # AuthType Basic + + # #Allow any valid user + # require valid-user + + RewriteEngine on + RewriteCond %{REQUEST_FILENAME} !-f + # not needed in a subdir + # RewriteCond %{REQUEST_URI} !^/server-status$ + RewriteRule ^(.*)$ index.php [QSA,L] + + </Directory> + + <Directory "/var/www/imlcinode/public_html/api/"> + RewriteEngine on + RewriteCond %{REQUEST_FILENAME} !-f + RewriteRule ^(.*)$ index.php [QSA,L] + </Directory> + +</VirtualHost> \ No newline at end of file diff --git a/docker/containers/web-server/php/extra-php-config.ini b/docker/containers/web-server/php/extra-php-config.ini new file mode 100644 index 0000000000000000000000000000000000000000..674b4dd6106afe25c6fb1074b64d9b7d77e91256 --- /dev/null +++ b/docker/containers/web-server/php/extra-php-config.ini @@ -0,0 +1,23 @@ +; +; GENERATED BY docker/init.sh - template: ./templates/extra-php-config.ini - 80c23edaf568e2c36b9926fe2339e481 +; +[PHP] + +error_reporting=E_ALL +display_errors=1 + +; ---------------------------------------------------------------------- +; XDEBUG STUFF BELOW +; ---------------------------------------------------------------------- +; +; +; [xdebug] +; xdebug.mode=develop,debug +; ; xdebug.client_host=localhost +; xdebug.start_with_request=yes +; ; xdebug.start_with_request=trigger +; +; xdebug.log=/tmp/xdebug.log +; xdebug.discover_client_host = 1 +; ; xdebug.client_port=9003 +; xdebug.idekey="netbeans-xdebug" \ No newline at end of file diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml new file mode 100644 index 0000000000000000000000000000000000000000..2b983d57879e787de6170039f05c3bd7b3fee8db --- /dev/null +++ b/docker/docker-compose.yml @@ -0,0 +1,57 @@ +# +# GENERATED BY docker/init.sh - template: ./templates/docker-compose.yml - 97c88229bd2b5099544c013052b8d9c3 +# +# ====================================================================== +# +# (1) see .env for set variables +# (2) run "docker-compose up" to startup +# +# ====================================================================== +version: '3.9' + +networks: + imlcinode-network: + +services: + + # ----- apache httpd + php + imlcinode-web-server: + + build: + context: . + dockerfile: ./containers/web-server/Dockerfile + + # keep "FROM" in docker file ... then image is not needed here + # image: "php:8.1-apache" + + container_name: 'imlcinode-server' + ports: + - '${APP_PORT}:80' + + working_dir: ${WEBROOT} + + volumes: + # service config + - ./containers/web-server/apache/sites-enabled:/etc/apache2/sites-enabled + - ./containers/web-server/php/extra-php-config.ini:/usr/local/etc/php/conf.d/extra-php-config.ini + + # data dirs + - ../data/.ssh:/var/www/.ssh + - ../data/imldeployment:/var/imldeployment + - ../data/tmp:/var/tmp/imldeployment + + # app webroot + - ../:/var/www/${APP_NAME} + + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost"] + interval: 10s + timeout: 3s + retries: 5 + # start_period: 40s + + networks: + - imlcinode-network + + user: ${DOCKER_USER_UID} + diff --git a/docker/init.sh b/docker/init.sh new file mode 100644 index 0000000000000000000000000000000000000000..3a073229be39e2a0bff88983946ded8f0bfad1a6 --- /dev/null +++ b/docker/init.sh @@ -0,0 +1,315 @@ +#!/bin/bash +# ====================================================================== +# +# DOCKER PHP DEV ENVIRONMENT :: INIT +# (work in progress) +# +# ---------------------------------------------------------------------- +# 2021-11-nn <axel.hahn@iml.unibe.ch> +# ====================================================================== + +cd $( dirname $0 ) +. $( basename $0 ).cfg + +# git@git-repo.iml.unibe.ch:iml-open-source/docker-php-starterkit.git +selfgitrepo="docker-php-starterkit.git" + +# ---------------------------------------------------------------------- +# FUNCTIONS +# ---------------------------------------------------------------------- + +# draw a headline 2 +function h2(){ + echo + echo -e "\e[33m>>>>> $*\e[0m" +} + +# draw a headline 3 +function h3(){ + echo + echo -e "\e[34m----- $*\e[0m" +} + +# function _gitinstall(){ +# h2 "install/ update app from git repo ${gitrepo} in ${gittarget} ..." +# test -d ${gittarget} && ( cd ${gittarget} && git pull ) +# test -d ${gittarget} || git clone -b ${gitbranch} ${gitrepo} ${gittarget} +# } + +# set acl on local directory +function _setWritepermissions(){ + h2 "set write permissions on ${gittarget} ..." + + local _user=$( id -gn ) + typeset -i local _user_uid=0 + test -f /etc/subuid && _user_uid=$( grep $_user /etc/subuid 2>/dev/null | cut -f 2 -d ':' )-1 + typeset -i local DOCKER_USER_OUTSIDE=$_user_uid+$DOCKER_USER_UID + + set -vx + + for mywritedir in ${WRITABLEDIR} + do + + echo "--- ${mywritedir}" + # remove current acl + sudo setfacl -bR "${mywritedir}" + + # default permissions: both the host user and the user with UID 33 (www-data on many systems) are owners with rwx perms + sudo setfacl -dRm u:${DOCKER_USER_OUTSIDE}:rwx,${_user}:rwx "${mywritedir}" + + # permissions: make both the host user and the user with UID 33 owner with rwx perms for all existing files/directories + sudo setfacl -Rm u:${DOCKER_USER_OUTSIDE}:rwx,${_user}:rwx "${mywritedir}" + done + + set +vx +} + +# cleanup starterkit git data +function _removeGitdata(){ + h2 "Remove git data of starterkit" + echo -n "Current git remote url: " + git config --get remote.origin.url + git config --get remote.origin.url 2>/dev/null | grep $selfgitrepo >/dev/null + if [ $? -eq 0 ]; then + echo + echo -n "Delete local .git and .gitignore? [y/N] > " + read answer + test "$answer" = "y" && ( echo "Deleting ... " && rm -rf ../.git ../.gitignore ) + else + echo "It was done already - $selfgitrepo was not found." + fi + +} + +# helper function: cut a text file starting from database start marker +# see _generateFiles() +function _fix_no-db(){ + local _file=$1 + if [ $DB_ADD = false ]; then + typeset -i local iStart=$( cat ${_file} | fgrep -n "$CUTTER_NO_DATABASE" | cut -f 1 -d ':' )-1 + if [ $iStart -gt 0 ]; then + sed -ni "1,${iStart}p" ${_file} + fi + fi +} + +# loop over all files in templates subdir make replacements and generate +# a target file. +# It skips if +# - 1st line is not starting with "# TARGET: filename" +# - target file has no updated lines +function _generateFiles(){ + + # re-read config vars + . $( basename $0 ).cfg + + local _tmpfile=/tmp/newfilecontent$$.tmp + h2 "generate files from templates..." + for mytpl in $( ls -1 ./templates/* ) + do + # h3 $mytpl + local _doReplace=1 + + # fetch traget file from first line + target=$( head -1 $mytpl | grep "^# TARGET:" | cut -f 2- -d ":" | awk '{ print $1 }' ) + + if [ -z "$target" ]; then + echo SKIP: $mytpl - target was not found in 1st line + _doReplace=0 + fi + + # write generated files to target + if [ $_doReplace -eq 1 ]; then + + # write file from line 2 to a tmp file + sed -n '2,$p' $mytpl >$_tmpfile + + # add generator + # sed -i "s#{{generator}}#generated by $0 - template: $mytpl - $( date )#g" $_tmpfile + local _md5=$( md5sum $_tmpfile | awk '{ print $1 }' ) + sed -i "s#{{generator}}#GENERATED BY $0 - template: $mytpl - $_md5#g" $_tmpfile + + # loop over vars to make the replacement + grep "^[a-zA-Z]" $( basename $0 ).cfg | while read line + do + # echo replacement: $line + mykey=$( echo $line | cut -f 1 -d '=' ) + myvalue="$( eval echo \"\${$mykey}\" )" + # grep "{{$mykey}}" $_tmpfile + + # TODO: multiline values fail here in replacement with sed + sed -i "s#{{$mykey}}#${myvalue}#g" $_tmpfile + done + _fix_no-db $_tmpfile + + # echo "changes for $target:" + diff "../$target" "$_tmpfile" | grep -v "$_md5" | grep -v "^---" | grep . + if [ $? -eq 0 -o ! -f "../$target" ]; then + echo -n "$mytpl - changes detected - writing [$target] ... " + mkdir -p $( dirname "../$target" ) || exit 2 + mv "$_tmpfile" "../$target" || exit 2 + echo OK + else + rm -f $_tmpfile + echo "SKIP: $mytpl - Nothing to do." + fi + fi + echo + done + +} + +# loop over all files in templates subdir make replacements and generate +# a traget file. +function _removeGeneratedFiles(){ + h2 "remove generated files..." + for mytpl in $( ls -1 ./templates/* ) + do + h3 $mytpl + + # fetch traget file from first line + target=$( head -1 $mytpl | grep "^# TARGET:" | cut -f 2- -d ":" | awk '{ print $1 }' ) + + if [ ! -z "$target" -a -f "../$target" ]; then + echo -n "REMOVING " + ls -l "../$target" || exit 2 + rm -f "../$target" || exit 2 + echo OK + else + echo SKIP: $target + fi + + done +} + +function _showContainers(){ + local bLong=$1 + h2 CONTAINERS + if [ -z "$bLong" ]; then + docker-compose ps + else + docker ps | grep $APP_NAME + fi +} + + +# a bit stupid ... i think I need to delete it. +function _showInfos(){ + _showContainers long + h2 INFO + + h3 "processes" + docker-compose top + + h3 "Check app port" + >/dev/tcp/localhost/${APP_PORT} 2>/dev/null && ( + echo "OK, app port ${APP_PORT} is reachable" + echo + echo "In a web browser open:" + echo " $frontendurl" + ) + h3 "Check database port" + >/dev/tcp/localhost/${DB_PORT} 2>/dev/null && ( + echo "OK, db port ${DB_PORT} is reachable" + echo + echo "In a local DB admin tool:" + echo " host : localhost" + echo " port : ${DB_PORT}" + echo " user : root" + echo " password: ${MYSQL_ROOT_PASS}" + ) + echo +} + +function _wait(){ + echo -n "... press RETURN > "; read dummy +} + +# ---------------------------------------------------------------------- +# MAIN +# ---------------------------------------------------------------------- + +action=$1 + +while true; do + echo + echo -e "\e[32m===== INITIALIZER FOR APP [$APP_NAME] ===== \e[0m" + + if [ -z "$action" ]; then + + _showContainers + + h2 MENU + echo " g - remove git data of starterkit" + echo + echo " i - init application: set permissions" + echo " t - generate files from templates" + echo " T - remove generated files" + echo + echo " u - startup containers docker-compose up -d" + echo " s - shutdown containers docker-compose stop" + echo " r - remove containers docker-compose rm -f" + echo + echo " m - more infos" + echo " c - console (bash)" + echo + echo -n " select >" + read action + fi + + case "$action" in + g) + _removeGitdata + ;; + i) + # _gitinstall + _setWritepermissions + ;; + t) + _generateFiles + ;; + T) + _removeGeneratedFiles + rm -rf containers + ;; + f) + _removeGeneratedFiles + _generateFiles + _wait + ;; + m) + _showInfos + _wait + ;; + u) + if docker-compose --verbose up -d --remove-orphans --build; then + # test ! -z "${APP_ONSTARTUP}" && sleep 2 && docker exec -it appmonitor-server /bin/bash -c "${APP_ONSTARTUP}" + echo "In a web browser:" + echo " $frontendurl" + else + echo "ERROR: docker-compose up failed :-/" + docker-compose logs | tail + fi + echo + + _wait + ;; + s) + docker-compose stop + ;; + r) + docker-compose rm -f + ;; + c) + docker ps + echo -n "id or name >" + read dockerid + test -z "$dockerid" || docker exec -it $dockerid /bin/bash + ;; + *) echo "ACTION [$action] NOT IMPLEMENTED." + esac + action= +done + + +# ---------------------------------------------------------------------- diff --git a/docker/init.sh.cfg b/docker/init.sh.cfg new file mode 100644 index 0000000000000000000000000000000000000000..2be3a4dbe45a5097d2cb5ded81fb4d2f444c2fa3 --- /dev/null +++ b/docker/init.sh.cfg @@ -0,0 +1,76 @@ +# ====================================================================== +# +# settings for init.sh and base values for replacements in template files +# This script is sourced by init.sh ... this file is bash syntax +# +# ---------------------------------------------------------------------- +# 2021-11-xx <axel.hahn@iml.unibe.ch> +# ====================================================================== + +APP_NAME=imlcinode + +# web port 80 in container is seen on localhost as ... +APP_PORT=8002 + +# document root inside container +WEBROOT=/var/www/${APP_NAME}/public_html + +APP_APT_PACKAGES="git unzip zip rsync" +# TODO: +# APP_APT_PACKAGES="git unzip zip yui-compressor" +# uglify-js yarn +# libpq-dev openjdk-7-jdk maven zlib1g-dev + +APP_APACHE_MODULES="rewrite" +# APP_APACHE_MODULES="" + +APP_PHP_VERSION=8.1 +# sqlite3 is active already +APP_PHP_MODULES="curl mbstring ldap intl xml" + +# ONSTARTUP="docker exec -it appmonitor-server nohup /usr/local/bin/php /var/www/appmonitor/public_html/server/service.php > /tmp/appmonitor-service.log &" +# APP_ONSTARTUP="php ${WEBROOT}/server/service.php" +# APP_ONSTARTUP="a2enmod rewrite" + + +# ---------------------------------------------------------------------- + +# add a container with database? +DB_ADD=false + +# ---------------------------------------------------------------------- +# for an optional database server + +DB_PORT=13306 + +# ----- database settings +MYSQL_IMAGE=mariadb:10.5.9 +MYSQL_RANDOM_ROOT_PASSWORD=0 +MYSQL_ALLOW_EMPTY_PASSWORD=0 +MYSQL_ROOT_PASS=12345678 +MYSQL_USER=${APP_NAME} +MYSQL_PASS=mypassword +MYSQL_DB=${APP_NAME} + + +# ====================================================================== +# ignore things below + + +# where to set acl where local user and web user in container +# can write simultanously +# divide multiple dirs with space +WRITABLEDIR="../public_html ../config ../data" + + +# web service user in container +DOCKER_USER_UID=33 + +# document root inside web-server container +WEBROOT=/var/www/${APP_NAME}/public_html + +CUTTER_NO_DATABASE="CUT-HERE-FOR-NO-DATABASE" + +frontendurl=http://localhost:${APP_PORT}/ + +# ---------------------------------------------------------------------- diff --git a/docker/templates/docker-compose.yml b/docker/templates/docker-compose.yml new file mode 100644 index 0000000000000000000000000000000000000000..0cdc088a13e05801f43a11cfab9881c8c3158075 --- /dev/null +++ b/docker/templates/docker-compose.yml @@ -0,0 +1,84 @@ +# TARGET: docker/docker-compose.yml +# +# {{generator}} +# +# ====================================================================== +# +# (1) see .env for set variables +# (2) run "docker-compose up" to startup +# +# ====================================================================== +version: '3.9' + +networks: + {{APP_NAME}}-network: + +services: + + # ----- apache httpd + php + {{APP_NAME}}-web-server: + + build: + context: . + dockerfile: ./containers/web-server/Dockerfile + + # keep "FROM" in docker file ... then image is not needed here + # image: "php:{{APP_PHP_VERSION}}-apache" + + container_name: '{{APP_NAME}}-server' + ports: + - '${APP_PORT}:80' + + working_dir: ${WEBROOT} + + volumes: + # service config + - ./containers/web-server/apache/sites-enabled:/etc/apache2/sites-enabled + - ./containers/web-server/php/extra-php-config.ini:/usr/local/etc/php/conf.d/extra-php-config.ini + + # data dirs + - ../data/.ssh:/var/www/.ssh + - ../data/imldeployment:/var/imldeployment + - ../data/tmp:/var/tmp/imldeployment + + # app webroot + - ../:/var/www/${APP_NAME} + + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost"] + interval: 10s + timeout: 3s + retries: 5 + # start_period: 40s + + networks: + - {{APP_NAME}}-network + + user: ${DOCKER_USER_UID} + + # --- 8< --- {{CUTTER_NO_DATABASE}} --- 8< --- + + depends_on: + - {{APP_NAME}}-db-server + + # ----- mariadb + {{APP_NAME}}-db-server: + image: {{MYSQL_IMAGE}} + container_name: '${APP_NAME}-db' + # restart: always + ports: + - '${DB_PORT}:3306' + environment: + MYSQL_ROOT_PASSWORD: '${MYSQL_ROOT_PASS}' + MYSQL_USER: '${MYSQL_USER}' + MYSQL_PASSWORD: '${MYSQL_PASS}' + MYSQL_DATABASE: '${MYSQL_DB}' + volumes: + # - ./containers/db-server/db_data:/var/lib/mysql + - ./containers/db-server/mariadb/my.cnf:/etc/mysql/conf.d/my.cnf + healthcheck: + test: mysqladmin ping -h 127.0.0.1 -u root --password=$$MYSQL_ROOT_PASSWORD + interval: 5s + retries: 5 + networks: + - {{APP_NAME}}-network diff --git a/docker/templates/dot_env b/docker/templates/dot_env new file mode 100644 index 0000000000000000000000000000000000000000..bc8af1d5372ca0731b4e2eebd0a3e95cc2d1c78b --- /dev/null +++ b/docker/templates/dot_env @@ -0,0 +1,28 @@ +# TARGET: docker/.env +# ====================================================================== +# +# {{generator}} +# values to be used in docker-composer.yml +# +# ====================================================================== + +# ----- application +APP_NAME={{APP_NAME}} + +# uid of www-data in the docker container +DOCKER_USER_UID={{DOCKER_USER_UID}} + +APP_PORT={{APP_PORT}} +WEBROOT={{WEBROOT}} + +# --- 8< --- {{CUTTER_NO_DATABASE}} --- 8< --- + +DB_PORT={{DB_PORT}} + +# ----- database settings +MYSQL_RANDOM_ROOT_PASSWORD={{MYSQL_RANDOM_ROOT_PASSWORD}} +MYSQL_ALLOW_EMPTY_PASSWORD={{MYSQL_ALLOW_EMPTY_PASSWORD}} +MYSQL_ROOT_PASS={{MYSQL_ROOT_PASS}} +MYSQL_USER={{APP_NAME}} +MYSQL_PASS={{MYSQL_PASS}} +MYSQL_DB={{APP_NAME}} diff --git a/docker/templates/extra-php-config.ini b/docker/templates/extra-php-config.ini new file mode 100644 index 0000000000000000000000000000000000000000..689921c061d9bb2787a44c8e8807ef184016a69d --- /dev/null +++ b/docker/templates/extra-php-config.ini @@ -0,0 +1,24 @@ +# TARGET: docker/containers/web-server/php/extra-php-config.ini +; +; {{generator}} +; +[PHP] + +error_reporting=E_ALL +display_errors=1 + +; ---------------------------------------------------------------------- +; XDEBUG STUFF BELOW +; ---------------------------------------------------------------------- +; +; +; [xdebug] +; xdebug.mode=develop,debug +; ; xdebug.client_host=localhost +; xdebug.start_with_request=yes +; ; xdebug.start_with_request=trigger +; +; xdebug.log=/tmp/xdebug.log +; xdebug.discover_client_host = 1 +; ; xdebug.client_port=9003 +; xdebug.idekey="netbeans-xdebug" \ No newline at end of file diff --git a/docker/templates/readme.md b/docker/templates/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..34c2c57c1ad0518f5cc00d12148cdddc0720187f --- /dev/null +++ b/docker/templates/readme.md @@ -0,0 +1,7 @@ +# Templates + +## Rules + +* in the first line must be a line `# TARGET: [name of target file]` to define the target file +* Placeholdrs have the syntax variable in double brackets, i.e. `{{VARNAME}}` +* variables to be replaced are those in docker/init.sh.cfg and `{{genrator}}` diff --git a/docker/templates/vhost_app.conf b/docker/templates/vhost_app.conf new file mode 100644 index 0000000000000000000000000000000000000000..ea818a94da9d64b026f37f995693a29194d868c9 --- /dev/null +++ b/docker/templates/vhost_app.conf @@ -0,0 +1,37 @@ +# TARGET: docker/containers/web-server/apache/sites-enabled/vhost_app.conf +# +# {{generator}} +# + + +# load modules +# LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so + + +<VirtualHost *:80> + DocumentRoot {{WEBROOT}} + + <Directory "{{WEBROOT}}/deployment/"> + + # AuthUserFile {{WEBROOT}}/deployment/.htusers + # AuthName "IML DEPLOYMENT GUI" + # AuthType Basic + + # #Allow any valid user + # require valid-user + + RewriteEngine on + RewriteCond %{REQUEST_FILENAME} !-f + # not needed in a subdir + # RewriteCond %{REQUEST_URI} !^/server-status$ + RewriteRule ^(.*)$ index.php [QSA,L] + + </Directory> + + <Directory "{{WEBROOT}}/api/"> + RewriteEngine on + RewriteCond %{REQUEST_FILENAME} !-f + RewriteRule ^(.*)$ index.php [QSA,L] + </Directory> + +</VirtualHost> \ No newline at end of file diff --git a/docker/templates/web-server-Dockerfile b/docker/templates/web-server-Dockerfile new file mode 100644 index 0000000000000000000000000000000000000000..d2443af6363a7d5ad2194d9c02349fb8fc3e218d --- /dev/null +++ b/docker/templates/web-server-Dockerfile @@ -0,0 +1,15 @@ +# TARGET: docker/containers/web-server/Dockerfile +# +# {{generator}} +# +FROM php:{{APP_PHP_VERSION}}-apache + +# install packages +RUN apt-get update && apt-get install -y {{APP_APT_PACKAGES}} + +# enable apache modules +RUN a2enmod {{APP_APACHE_MODULES}} + +# install php packages +COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/ +RUN install-php-extensions {{APP_PHP_MODULES}} diff --git a/docs/10_Introduction/10_Installation_on_a_server.md b/docs/10_Introduction/10_Installation_on_a_server.md new file mode 100644 index 0000000000000000000000000000000000000000..a5e941dab30672f20f604dc11d82eb7091d2ed6e --- /dev/null +++ b/docs/10_Introduction/10_Installation_on_a_server.md @@ -0,0 +1,135 @@ +# Installation of CISERVER # + +You can install the CISERVER on your own host. You need full access to the system - it won't run on a shared hosting. + +## Apache Httpd + PHP ## + +Install an Apache httpd and enablethese modules. + +- rewrite +- proxy + proxy_fcgi (or proxy_http) for a proxy +- socache_shmcb (on Debian for ssl connections) + +For PHP 8.1 we need these packages + +- php-fpm +- php-curl +- php-intl +- php-mbstring +- php-ldap +- php-sqlite3 +- php-xml + +## Other required tools ## + +These commandline tools must be installed. + +- ssh +- rsync +- git + +## Get sources ## + +Extract the repository in `/var/www/ciserver.example.org`. +You can download the archive from the git repository or use `git clone`. + +```txt +cd /var/www +git clone https://git-repo.iml.unibe.ch/iml-open-source/imldeployment.git +mv imldeployment ciserver.example.com +``` + +The directory `/var/www/ciserver.example.com` is called approot in further documentation. + +## Update virtual host config ## + +Set the document root to the subdir `public_html`. +We need two rewrite rules to redirect requests. + +```txt +... +DocumentRoot "/var/www/ciserver.example.com/public_html" + +<location "/deployment/"> + RewriteEngine on + RewriteCond %{REQUEST_FILENAME} !-f + RewriteRule ^(.*)$ index.php [QSA,L] +</Location> + +<Location "/api/"> + RewriteEngine on + RewriteCond %{REQUEST_FILENAME} !-f + RewriteRule ^(.*)$ index.php [QSA,L] +</Location> +... +``` + +## Create Configs + +In `[approot]/config/` copy the 2 *.dist files to the same filename but without ".dist". + +## Create data structure and tmp + +The aplication works with + +- a data directory /var/imldeployment +- a tmp directory /var/tmp/imldeployment + +If you use ansible you can use this snippet. + +```txt +- name: extra appdirs + become: yes + become_user: root + hosts: ciserver + + tasks: + - name: Create CI SERVER base dirs + file: + path: '{{ item }}' + mode: 0750 + owner: www-data + group: www-data + state: directory + loop: + - '/var/tmp/imldeployment' + - '/var/imldeployment' + - '/var/imldeployment/data' + - '/var/imldeployment/data/database' + - '/var/imldeployment/data/projects' + - '/var/imldeployment/data/sshkeys' + - '/var/imldeployment/build' + - '/var/imldeployment/defaults' + - '/var/imldeployment/packages' + - '/var/imldeployment/packages/_files' +``` + +## Enable shell for Apache service user ## + +The service user of the webservice needs to execute commands with php function exec. By default this user has set nologin as shell - this muust be changed to `bin/bash`. + +Remark: the username can differ from distribution to distribution. Maybe it is "apache" or "wwwrun" on your system. + +In the /etc/passwd edit the line of "www-data": + +```txt +... +www-data:x:33:33:www-data:/home/www-data:/bin/bash +... +``` + +It can be a good idea to switch the $HOME from /var/www to the standard directory for users `/home/wwww-data` too. + +As Ansible snippet (remark: changing $HOME works if the user has no process - maybe you need to stop php-fpm and apache service) + +```txt + - name: give a shell to www-data + ansible.builtin.user: + name: www-data + home: /home/www-data + shell: /bin/bash +``` + +## First check + +Open `check-config.php`in the webroot, i.e. <https://ciserver.example.com/check-config.php>. diff --git a/docs/10_Introduction/20_Installation_with_Docker.md b/docs/10_Introduction/20_Installation_with_Docker.md new file mode 100644 index 0000000000000000000000000000000000000000..5633cd4b69baaad29201ec990558a836879ad896 --- /dev/null +++ b/docs/10_Introduction/20_Installation_with_Docker.md @@ -0,0 +1,114 @@ +# Installation with a local Docker service # + +For development a docker environment is part of the repository data. + +## Requirements ## + +* Linux system +* a running rootless Docker service +* tools/ packages: + * Git + * Docker-compose + * facl to set ACL for write permissions for your user and the webservice of the container +* sudo permssions + * to set ACL permissions with setfacl + * to remove tmp data + +## Get Sources ## + +As your local user execute the following steps: + +```bash +cd [somewhere] +git https://git-repo.iml.unibe.ch/iml-open-source/imldeployment.git +cd imldeployment +``` + +## Set permissions ## + +In the folder `docker` are all configurations and helpers to run a docker container. +In it is an `init.sh` to set environment. + +```shell +./docker/init.sh +``` + +You get a menu. + +```txt +>>>>> MENU + g - remove git data of starterkit + + i - init application: set permissions + t - generate files from templates + T - remove generated files + + u - startup containers docker-compose up -d + s - shutdown containers docker-compose stop + r - remove containers docker-compose rm -f + + m - more infos + c - console (bash) + + select > +``` + +Then press `i` and `Return` to set permissions. + +This sets the acl permissions for the subdirs + +* public_html +* config +* data + +You will see something like that: + +```txt ++ for mywritedir in ${WRITABLEDIR} ++ echo '--- ../public_html' +--- ../public_html ++ sudo setfacl -bR ../public_html ++ sudo setfacl -dRm u:231104:rwx,axel:rwx ../public_html ++ sudo setfacl -Rm u:231104:rwx,axel:rwx ../public_html ++ for mywritedir in ${WRITABLEDIR} ++ echo '--- ../config' +--- ../config ++ sudo setfacl -bR ../config ++ sudo setfacl -dRm u:231104:rwx,axel:rwx ../config ++ sudo setfacl -Rm u:231104:rwx,axel:rwx ../config ++ for mywritedir in ${WRITABLEDIR} ++ echo '--- ../data' +--- ../data ++ sudo setfacl -bR ../data ++ sudo setfacl -dRm u:231104:rwx,axel:rwx ../data ++ sudo setfacl -Rm u:231104:rwx,axel:rwx ../data ++ set +vx +``` + +## Start container ## + +Then press `u` and `Return` to run `docker-compuse up`. +On the 1st run it needs to download the PHP docker image with Apache httpd and takes a few more seconds. + +If ist is up you can run <http://localhost:8002/> in your webbrowser. + +## Change port ## + +If you need to change the port then stop a running container. +Edit `docker/init.sh.cfg` and set a new port + +```txt +... +# web port 80 in container is seen on localhost as ... +APP_PORT=8002 +... +``` + +After any change in init.sh.cfg we update the configs with + +```shell +./docker/init.sh +``` + +Then press `t` (generate files from templates) + `Return`. +If you start the container again the application is available under the new port. diff --git a/docs/10_Introduction/30_Filestructure.md b/docs/10_Introduction/30_Filestructure.md new file mode 100644 index 0000000000000000000000000000000000000000..000c6a505415e0c5557fe79617396803c87e31b1 --- /dev/null +++ b/docs/10_Introduction/30_Filestructure.md @@ -0,0 +1,77 @@ +# File structure # + +* web - ui and api +* data dir - configuration, database, built archives +* temp area - checked out projects to read comit messages + +## Approot and website ## + +Default: /var/www/[YOUR-DOMAIN]/ + +```txt +. +├── api +├── appmonitor +│ ├── classes +│ └── plugins +│ └── checks +├── ~cache +├── deployment +│ ├── classes +│ │ └── tests +│ ├── images +│ ├── js +│ ├── pages +│ └── plugins +│ ├── build +│ │ └── tgz +│ └── rollout +│ ├── awx +│ ├── default +│ └── ssh +├── valuestore +│ ├── classes +│ ├── data +│ └── tests +├── vendor +│ ├── bootstrap3 +│ ├── font-awesome +│ ├── font-awesome-4.7.0 +│ │ ├── css +│ │ └── fonts +│ ├── jquery +│ │ ├── 3.3.1 +│ │ └── 3.4.1 +│ ├── medoo +│ │ └── src +│ ├── shooker +│ ├── spyc +│ └── vis +│ └── 4.21.0 +│ └── img +│ └── network +├── versions +│ ├── classes +│ └── data +└── webservice +``` + +## Data ## + +By default: /var/imldeployment + +```txt +imldeployment/ +├── build << build directories +├── data +│ ├── database +│ ├── projects +│ └── sshkeys +├── defaults +└── packages << output data of buils + └── _files +``` + +## Temp ## + +By default: /var/tmp/imldeployment diff --git a/docs/10_Introduction/40_Dependencies.md b/docs/10_Introduction/40_Dependencies.md new file mode 100644 index 0000000000000000000000000000000000000000..b07292685e6a5c9a8003078e4c6a15fc05dea97e --- /dev/null +++ b/docs/10_Introduction/40_Dependencies.md @@ -0,0 +1,8 @@ + +# Dependencies # + +Related Components of the CI server + +* VCS +* Sync build packages +* Deploy targets diff --git a/docs/20_Server/Processes/10_Build.md b/docs/20_Server/Processes/10_Build.md new file mode 100644 index 0000000000000000000000000000000000000000..751827b272bbfa8a57393b6accbd90cefc329cd5 --- /dev/null +++ b/docs/20_Server/Processes/10_Build.md @@ -0,0 +1,143 @@ +# Build # + +A build process can be started ... + +* by `Build`button on overview page +* by `Build`button in application view +* by API call + +Among its steps are some builtin, some depend on the project settings and some can be influenced by the developers. + +A build is denied if a project has no activated phase the project settings. + +## Overview ## + + + +## Steps ## + +### Create working directory ### + +Outdated kept builds will be cleaned up. + +Below `/var/imldeployment/build/` a subdirectory with the id of the project will be created. In that one a uniq directory will be created by using a timestamp. + +### Get sources ### + +The `git clone` fetches the sources from the Git repository + +Remark: at the moment ony Git is suported as version control system. An interface in the classes subdir describes the required method to be implemented. An additional VCS will be available in the project settings if it was added. + +### Project specific actions ### + +An `chmod 755 /hooks/on*` makes hook scripts executable. + +These steps will be executed if they exist: + +* `hooks/onbuild-postclone` - actions after cloning the sources +* Sync other default files that are not in the repository - it was needed for rarely special projects. +* `hooks/onbuild` - actions before runinng the build or compression + +Recommendation: + +Use a script named `hooks/onbuild` and put your project specisif actions into it. +Read the next chapters to use given environment and scripts. + +#### Custom vars #### + +A file named `ci-custom-vars` will be created containing the given data in the project config. Its idea is to be sourced by hook script to set variables in the current shell. + +Example: + +If it contains + +```shell +export myVar="hello world" +``` + +it can be sourced with this snippet: + +```shell +cd `dirname $0` +cd .. +. ci-custom-vars || exit 1 +echo myVar=$myVar +``` + +#### Builtin environment #### + +* $GIT_SSH - full path to git ssh wrapper (CI-Root/shellscripts/gitsshwrapper.sh) +* $DIR_SSH_KEYS - full path to ssh keys (/var/imldeployment/data/sshkeys) +* $DIR_APPROOT - full path of the current build directory (z.B. /var/imldeployment/build/ci-webgui/ci-webgui_20171211-102707) +* $RVMSCRIPT - for Rails projects: path to the RVM script. With it you can set a custom Ruby version +* $NVMINIT - for NodeJs projects - to install a custom node version + +Snippets: + +(1) + +Set a Ruby version: +Rvm must be installed on the server where the ci server runs. + +```shell +. $RVMSCRIPT || exit 1 +rvm use 2.2.3 +``` + +(2) + +Set a custom Nodejs version: +The NVM init script is part of the ci server. + +```shell +. $RVMSCRIPT || exit 1 +rvm use 2.2.3 +. $NVMINIT || exit 1 +nvm install [Version] +``` + +Important: at the end of the hook script uninstall it by using `nvmremove`. + + Ressources: + +* CI-Git-Repo .. nvm_init.sh: <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/-/blob/master/shellscripts/nvm_init.sh> +* NVM: <https://github.com/nvm-sh/> + +### Remove vcs data ### + +The version control data (`.git` directory in build root) will be removed. + +### Build or compress ### + +(1) +A metafile `[Projecd_ID].json` will be created in the build root. It will be used to identify the package or installation. It contains date, branch and commit message. + +(2) +WIP + +All available built plugins have the name of the subdirs below `public_html/deployment/plugins/build/`. + +A loop over available plugins will execute the activated build plugins to initialize one or builds. + +The output dir is `/var/imldeployment/packages/_files/[Project_ID]/`. Here are subdirectories with the timestamp when the build was started, e.g. `./20220705_115145/` + +(3) +If there are files in hooks/templates/ the will be copied into the package output directory. They can be used by automation tools like puppet to generate configuration files for different targets and different phases. + +### Remove build dir ### + +If all actions were successful the buld directory will be deleted. + +### Add in queue of 1st phase ### + +A successful build triggers the queuing to the first active phase of this project. + +A phase can define deploy times to define, when a rollout is allowed. In the special case that the deploy time has no limit it will be installed instantly. Otherwise you will see the package in the queue column of a phase. + +## If a build fails ## + +The working directory will be kept. In the project config the number of kept projects is set (default: 3). Additionally old failed builds (older 7 days) will be deleted by a cronjob. + +If you open the project in the web ui you find the kept folders in the tab "Build errors". + +A sysadmin with access to the ci server can switch to the build directory and repeat the execution of a hook script. diff --git a/docs/20_Server/Processes/20_Rollout.md b/docs/20_Server/Processes/20_Rollout.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/20_Server/Usage/10_Login.md b/docs/20_Server/Usage/10_Login.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/20_Server/Usage/20_Project_overview.md b/docs/20_Server/Usage/20_Project_overview.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/_index.md b/docs/_index.md new file mode 100644 index 0000000000000000000000000000000000000000..bb1eb809e514615a3fdc328b6efe7a0076664f80 --- /dev/null +++ b/docs/_index.md @@ -0,0 +1,65 @@ +# CI Server # + +Free software and Open Source from University of Bern :: IML - Institute of Medical Education + +📄 Source: <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment> \ +📜 License: GNU GPL 3.0 \ +📖 Docs: TODO + +- - - + +## Description ## + +CI node that checks out projects from git repositories and builds an deployable archive. +The archives can be synched to multiple deployment targets e.g. puppet master or a protected software archive. + +```mermaid +flowchart LR + subgraph CI server + CI(CI<br>deployment<br>web gui) --> |Build<br>package| PkgDir + PkgDir[Package<br>dir] + end + + PkgDir --> |rsync| Pkg1 + PkgDir --> |rsync| Pkg2 + PkgDir --> |rsync| Pkg3 + + Pkg1 + + Pkg1(CI package<br>server 1) --> |secure<br>download| DeployClient + Pkg2(CI package<br>server N) + Pkg3(Puppet master) + + + DeployClient --> |installs| ApplicationA(Application A) + DeployClient --> |installs| ApplicationB(Application B) +``` + +This project is related to + +* CI package server <https://git-repo.iml.unibe.ch/iml-open-source/ci-pkg> +* Deployment client written in bash <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment-client> + +## Features ## + +* checkout from git via SSH with multiple ssh keys (can be extended with a plugin) +* build has hooks to customize build process +* In our institute it builds projects written in + * PHP + * NodeJS - using NVM for custom Node versions + * Ruby - using RVM for custom Ruby versions +* sync built archives to deploy systems +* trigger rollout via ssh command or AWX API call (can be extended with a plugin) +* receives install status +* sends messages (email, Slack) +* API to start a build from somewhere, e.g. from a devops workplace or Gitlab server + +## Screenshots ## + +The overview over all projects is the starting page after login. It shows all projects and which build is rolled out to which phase. + + + +The project overview for a single project: + + diff --git a/docs/config.json b/docs/config.json new file mode 100644 index 0000000000000000000000000000000000000000..3a54d3b278197ecdafb75e6e0c4cf3ca770f162e --- /dev/null +++ b/docs/config.json @@ -0,0 +1,22 @@ +{ + "title": "IML CISERVER", + "author": "Axel Hahn", + "tagline": "Build and rollout projects", + + "html": { + "auto_toc": true, + "auto_landing": false, + "date_modified": false, + "jump_buttons": true, + "edit_on_github_": "iml-it/appmonitor/tree/master/docs", + "edit_on": { + "name": "Gitlab", + "basepath": "https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/tree/master/docs" + }, + "links": { + "GitHub Repo": "https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/" + }, + "theme": "daux-blue", + "search": true + } +} diff --git a/docs/images-sources/processes-build.drawio b/docs/images-sources/processes-build.drawio new file mode 100644 index 0000000000000000000000000000000000000000..0e762c78a3b84fb422f541a5a3347e4081b2fee9 --- /dev/null +++ b/docs/images-sources/processes-build.drawio @@ -0,0 +1 @@ +<mxfile host="Electron" modified="2022-07-21T12:42:31.740Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/19.0.3 Chrome/100.0.4896.160 Electron/18.3.5 Safari/537.36" etag="S7cbsLsyLaclOyHWU7L3" version="19.0.3" type="device"><diagram id="u_O4M5wh5bko4jYocOWh" name="Page-1">5Vxtc6M2EP41nmk/1APi1R9jX+4y07RNm5n27lNGBhmrwYgKkcT99ZVAYIMIJjUY25dL5pAQWHqe3dXuSvLEWGzevlAYr38hPgonQPPfJsanCQC6CcBE/Gr+Nq+ZaVZeEVDsy0a7ikf8L5KVmqxNsY+SSkNGSMhwXK30SBQhj1XqIKXktdpsRcLqp8YwQErFowdDtfYv7LN1XusCZ1d/h3CwLj5Zt2f5nQ0sGsuRJGvok9e9KuN2YiwoISy/2rwtUCjAK3DJn/v8zt2yYxRFrMsDW39tpm/RywO7Cxa/LX+2n+7Nnww3f80LDFM5Ytlbti0goCSNfCTeok+M+esaM/QYQ0/cfeWk87o124Tytg+Tddk2YZQ8owUJCc1eZcxsx4A2v7PCYVjURyRCZeMCZCBqnhHz1vJdsp+IMvT2LgJ6iSsXSEQ2iNEtbyIfME1JhZRFoMny645Z05L8rfdZncmGUEpTUL57Bzi/kJh/AH9HQRv5XP5kkVC2JgGJYHi7q53v+NB4adfmnpBYgvU3YmwrlQmmjFQ54nDR7Vf5fFb4JgpTqyh+etu/+WkrSw18iN62s8EHR1LqoRYQCr2GNECspZ3dzC5FIWT4pdqP3pnSFUX5A8VEoa+CUany2r5maFU2qkoilSGESxQ+kAQzTCJe7XFQEb8/FwqAuXm6rzVYEsbIZq/BTYgDcYMJoZiTlIU44p9SWEnRCSiblC/n/Y3FODZvgbDmU7JaYQ9NfcjgEiYoKa+eNjjCT6CmyFzBgeaAuSvrCzFOSCig7UWFHbeqwqalqLCtqRpsD6XA+igaPJQmgo6aqJtjqiJQVPGR95hNgA03QtijZSL+g7zNMsV8wj9qRntfo4MQJom87kG2y+lIyrarzk5lk5PItqni/Hininvk3whHSxgSAQj2qvhVhT0RTBXtpbGr2RCk+xZyWt2HTr5CP3OT0VUljtQI+egDwbyLpUxYoOaygBrZ+QDkU/veX+1Ftl19Ud2jycenvCeTm3I4R5hJ1a38Dh0du6Mszca0rrai9aKz3JRSGHlrHovVzCwvh3ww8yXlV4G4gpGfjTJIWiyvdtjy/g+70HPEYM0Ouxs6OKVNVuO1DUyE99Yn0itL/BP1JGJ79Zpm8L8mBuzsZwAGnCIeqxvAQwwYQzEwUxjINeNJ75WDEaRdwdoeG+siC6SC/ev3IvBGQ5LixCSooS+37U9fv303FJhjU2C5re6T9KOP9pfaoT8vd3tUH6no5p5KfMmcpHyAF+/3gHqmtCEWPa3fY4ycKd3FDN8m+yHDiTOlZlf1MEZVDzVzsKAIMiQ+TEgRoc84CuqRhI8p8hiRHehLfXyI3JXXOIF4LlquTqA++ujqA2YXpT4NSaLBUp5dVx8AOFKlskf5qOB2r0EsMi6Jwm8PKZfCGbkQyoeit2tKGxxrMZvzbqCWd3M7Jt5OJycFQvvGOkQwSmM1s0PRhrwIKz4NMLs2J8ce3UobagrhkheTuvryYNTVJEu1k8cHUrXYdoVsr9EF8Z3Z8p0QawwmTGdMJoAaVT2EaYBxNDFuVGNUupTcxjzDAP2Q/NirSerC2sAmyXGc6dgZZ8O+KqPUNYIC1mW5e0CNuTg6+Wwt4i0eXPWqHWcQVrlnoB3AUmD/PUWpQJ2LNND0RDiD0MtEhg92DRN0bTzMzoAHQ12WaYH56I2M5QwwzkbG2q4AR1c9V6MJfl23BsK/EIk9/P9cPAr8Y+ThFfakFpDo4pOjZZxQiH/T8sCsYRtpfe9Gf+CrKzRemoj9gHxclPwttvwNzgTnwfXNJiZcsDQGWahRmHBGZ0INpjMXtmesR3BN61jrWtPC5GnBVuMFsd+P8XhhIBE//VyriHiDqT8x6iOnkidnsZXLlOtRh0Nqt5nf04TURTf3k3tr5D1PhHio2b1/UkzRBhUhygWbK9PUCp+0cJKM8b1UU92+Nc93KGscEXGKaRNTlFw8+tZsdoboW+rkfI8CJHYr1uDmI2eTw6c0Gpx/5UxF/TzGBvt+ZhKbSKztlS5Y6WmDeX0ucZv2dzXNJcPta2lf1b+ifS2zjtPF3tHAMRbu1Sj6jpDnLP3aMGHEJGFeKEi68ECintEwRl8KstqzrtejGQWqhzXj2BMWx+X51PzGgsTi8320gmnIVO3gGKYeS+nVaYc5unYANeHRaqcoGsSzOgMurNG5sNQ5IzM/3KV9EecS8uQ3ZEh4ufI8nha/kxQ5Q8frPRPZA5mmUSXTbshfGU1H/gZzyGzVzLVozEdPTzYdCm5RHWUOaqC9kqbvg5LaAWOzKbnrNFCi64NxooYtNNlG3scdhAoZxyRh9KnmGtVEjH0gEZOVHhDFHBShn6261U59N8vbo49id94jOKr3bquzYumcaBxAsS9dizObXK4K1ESoIiHZYjWity9ZQuiqvtLAI5GHYpZMxRfCZA1rAuZ8nt+YheDVbdaAU4Jl1OxPQ4xuNZgf8+MLe7y4+/aZfBvC7jt8jNv/AA==</diagram></mxfile> \ No newline at end of file diff --git a/docs/images/processes-build.png b/docs/images/processes-build.png new file mode 100644 index 0000000000000000000000000000000000000000..b1f88335177c5ca7188fe72531635a7e69df2fe9 Binary files /dev/null and b/docs/images/processes-build.png differ diff --git a/docs/images/screenshot_overview_all_projects.png b/docs/images/screenshot_overview_all_projects.png new file mode 100644 index 0000000000000000000000000000000000000000..badedafcbb2f157b8490b9d1d51f4864af05cf7b Binary files /dev/null and b/docs/images/screenshot_overview_all_projects.png differ diff --git a/docs/images/screenshot_overview_project.png b/docs/images/screenshot_overview_project.png new file mode 100644 index 0000000000000000000000000000000000000000..1f6fc60b23ef5b77c9df12471bcd685e1120794d Binary files /dev/null and b/docs/images/screenshot_overview_project.png differ diff --git a/docs/style.css b/docs/style.css new file mode 100644 index 0000000000000000000000000000000000000000..909562cb91d2a8335f4470332ef1e07bdb2ee44c --- /dev/null +++ b/docs/style.css @@ -0,0 +1,176 @@ +/* + + patch css elements of daux.io blue theme + version 2022-05-13 + +*/ + + +/* ---------- vars ---------- */ + +:root{ + + /* background colors */ + --bg:none; + --bg-body: #fff; + --bg-navlinkactive:#f4f4f4; + --bg-navlinkactive: linear-gradient(-90deg,rgba(0,0,0,0), rgba(40,60,80,0.05) 30%); + --bg-pre:#f8f8f8; + --bg-toc: #fff; + + /* foreground colors */ + --color: #234; + --navlinkactive:#f33; + --title: #aaa; + + --link:#12a; + --toclink:rgba(40,60,80,0.8); + + --h1: rgba(40,60,80,0.8); + --h1-bottom: 1px solid rgba(40,60,80,0.1); + --h2: #468; + --h3: #579; + +} + +/* ---------- tags ---------- */ + +a.Brand::before { + background: rgb(255,0,51); + color: #fff; + font-family: arial; + font-weight: bold; + padding: 0.5em 0.3em; + content: 'IML'; + margin-right: 0.4em; +} + +body, *{color: var(--color);} +body{background: var(--bg-body);} + + +a{color: var(--link);} +a:hover{opacity: 0.7;} + +h1>a{ color:var(--title);} +_h1:nth-child(1){position: fixed; background: var(--bg); box-shadow: 0 0 1em #ccc; padding: 0 1em} +h1:nth-child(1)>a{ color:var(--navlinkactive); } + +.s-content h1{color: var(--h1); font-size: 200%; font-weight:bold; margin-top: 2em; border-bottom: var(--h1-bottom);} +.s-content h2{color: var(--h2); font-size: 160%; } +.s-content h3{color: var(--h3); font-size: 140%; } +.s-content h4{margin: 0; font-size: 100%; text-align: center; background-color: rgba(0,0,0,0.05);padding: 0.3em;} + +.s-content pre{ + background: var(--bg-pre); +} + +/* ---------- classes ---------- */ + +.required{color:#a42;} +.optional{color:#888;} + + +/* ----- top left */ +.Brand, +.Columns__left { + background: var(--bg); + border-right: 0px solid #e7e7e9; + color: var(--color); +} +.Brand{font-size: 200%; + background_: linear-gradient(-10deg,#fff 50%, #ddd); + background: var(--bg); +} +.Columns__right__content { + background: var(--bg); +} + +/* ----- Navi left */ + +.Nav a:hover{ + background: none; + color: var(--navlinkactive) !important; +} + +.Nav__item--active { + border-right_: 0.3em solid var(--navlinkactive); +} +.Nav__item--active > a{ + background: var(--bg-navlinkactive); + color: var(--navlinkactive); +} +.Nav .Nav .Nav__item--active a { + color: var(--navlinkactive); +} +.Nav .Nav .Nav__item a { + opacity: 1; +} +.Nav__item--open > a { + background-color: var(--bg); +} + +.Nav a[href*="__Welcome"]{ + background: url("/icons/house.png") no-repeat 10px 4px ; + padding-left: 40px; +} +.Nav a[href*="__How_does_it_work"]{ + background: url("/icons/light-bulb.png") no-repeat 10px 4px ; + padding-left: 40px; +} + + + + +/* ---------- classes ---------- */ + +/* FIX smaller fnt size in tables */ +.s-content table { + font-size: 1em; +} + + +/* TOC */ +@media(min-width:1700px){ + .TableOfContentsContainer{ + position: fixed; + right: 2em; + top: 1em; + } +} + +.TableOfContentsContainer{ + border-top-left-radius: 1em; + background-color: var(--bg-toc); + border-left: 2px solid rgba(0,0,0,0.05); + padding: 0em; +} +.TableOfContentsContainer__content { + + border: none; + font-size: 0.5em; + +} +ul.TableOfContents ul{ + list-style-type: none; + padding-left: 1em; +} +.TableOfContentsContainer a{ color:var(--toclink);} + +.TableOfContentsContainer__content > .TableOfContents > li + li { + border-top: none; +} +.TableOfContentsContainer__content > .TableOfContents > li { + border-bottom: 1px dashed #ddd; +} + +/* pager - prev .. next */ +.s-content{ + margin-bottom: 6em; +} +.Pager{ + border-top: 1px dashed #aaa; margin: 0; padding: 1em; +} +.Pager a{ + color:var(--navlinkactive); +} diff --git a/history.md b/history.md new file mode 100644 index 0000000000000000000000000000000000000000..1db24cca31e40830c233a242498bf59201fdcf42 --- /dev/null +++ b/history.md @@ -0,0 +1,27 @@ + +# HISTORY + +## 2022-08-nn + +🔵 ADDED: docker environment to move away from develop environment as a Virtual box installation +🔵 ADDED: Compatible to PHP 8.1 +🔵 ADDED: plugins for build process; first plugin: compress as tgz +🔵 ADDED: project groups. A project can set a project group. In the overview page is an additional filter of used groups + +🐞 FIX: log filter "all" did not work +🐞 FIX: inaccessible projects in top menu bar: added scrolling + +✅ UPDATE: rename steps in phases "Puppet" --> "Ready" (to install) + +- - - + +🔵 +🔆 New + +✅ Fix + +🐞 Bug + +🔥 💣 Critical bug + +🛡️ Security \ No newline at end of file diff --git a/hooks/onbuild b/hooks/onbuild index 868e19370d85bb75ea7bf9eff73e1afe710e3c30..a343328a460814adc0f4cf60f6297db4320e9d3a 100644 --- a/hooks/onbuild +++ b/hooks/onbuild @@ -7,14 +7,20 @@ # - Projekt-Configs anpassen # # 2014-05-07 axel.hahn@iml.unibe.ch +# 2022-07-20 axel.hahn@iml.unibe.ch remove unneeded dirs, e.g. docker # ====================================================================== +dirs2remove="data docker" + echo ONDEPLOY fuer CI Deployment GUI echo -cd `dirname $0` +cd $( dirname $0 ) cd .. -echo "----------> setze x-recht auf gitsshwrapper " +echo "----------> Set X permissions on gitsshwrapper " ls -l shellscripts/gitsshwrapper.sh && chmod 755 shellscripts/gitsshwrapper.sh && ls -l shellscripts/gitsshwrapper.sh +echo "----------> Reove unneeded dirs: $dirs2remove" +rm -rf ${dirs2remove} + echo "----------> done." diff --git a/hooks/templates/config_custom.php.erb b/hooks/templates/config_custom.php.erb new file mode 100644 index 0000000000000000000000000000000000000000..44582499e84d711218152a2f0a5c17dcfb12ff55 --- /dev/null +++ b/hooks/templates/config_custom.php.erb @@ -0,0 +1,115 @@ +<?php +/* + * TARGET: config/config_custom.php.php + * ---------------------------------------------------------------------- + * CUSTOM SETTINGS + * + * Settings here override defaults from config_defaults.php + * + */ +return [ + + 'lang' => '<%= @replace["lang"] %>', // for available languages see ./config/lang/*.json + + 'auth' => [ + // force using a given user ... for development only + 'forceuser' => false, + + // use a real login + 'ldap' => [ + 'server' => '<%= @replace["ldap-url"] %>', + 'port' => <%= @replace["ldap-port"] %>, + 'DnLdapUser' => '<%= @replace["ldap-user"] %>', + 'PwLdapUser' => '<%= @replace["ldap-password"] %>', + 'DnUserNode' => '<%= @replace["ldap-dn-user"] %>', + 'DnAppNode' => '<%= @replace["ldap-cn-apps"] %><%= @replace["ldap-dn-apps"] %>', + 'debugLevel' => 0, + ] + ], + 'banner' => <%= @replace["banner"] %>, + + // ---------------------------------------------------------------------- + + 'phases' => [ + "preview" => [], + "stage" => [], + "live" => [ + // prevent immediate installation after build or accept + "deploytimes" => ['<%= @replace["deploytimes-live"] %>'], + ], + ], + 'showdebug' => [ + 'ip'=> [<%= @replace["debug-ips"] %>], + ], + + 'projectgroups' => [<%= @replace["projectgroups"] %>], + + // ---------------------------------------------------------------------- + // build settings + // ---------------------------------------------------------------------- + 'versionsToKeep' => <%= @replace["versions-to-keep"] %>, // for cleanup: keep n unused versions + 'builtsToKeep' => <%= @replace["builds-to-keep"] %>, + 'build' => [ + 'env' => 'export RVMSCRIPT="/usr/local/rvm/scripts/rvm";', + 'hooks' => [ + 'build-postclone' => 'hooks/onbuild-postclone', + 'build-precompress' => 'hooks/onbuild', + ], + ], + + // ---------------------------------------------------------------------- + // sync of archives + // ---------------------------------------------------------------------- + 'mirrorPackages' => [<%= @replace["mirror-packages"] %>], + + // ---------------------------------------------------------------------- + // plugins + // existing subkeys = enabled plugins + // ---------------------------------------------------------------------- + 'plugins'=>[ + + 'rollout'=>[ + 'default'=>[], + 'ssh'=>[ + 'user'=>'<%= @replace["rollout-ssh-user"] %>', + 'privatekey'=>'', + 'addkeycommand'=>'/usr/bin/ssh-keygen -R %s; /usr/bin/ssh-keyscan -t rsa %s >> /home/www-data/.ssh/known_hosts', + 'testcommand'=>'<%= @replace["rollout-ssh-testcommand"] %>', + 'command'=>'<%= @replace["rollout-ssh-command"] %>', + ], + 'awx'=>[ + 'url'=>'<%= @replace["rollout-axw-url"] %>', // no ending "/" + 'user'=>'<%= @replace["rollout-axw-user"] %>', + 'password'=>'<%= @replace["rollout-axw-password"] %>', + 'jobtemplate'=>'<%= @replace["rollout-axw-jobtemplate"] %>', + 'tags'=>'<%= @replace["rollout-axw-tags"] %>', + // 'ignore-ssl-error'=>false, + ], + ], + ], + + // ---------------------------------------------------------------------- + // notifications to messengers ... + // ---------------------------------------------------------------------- + + // notifications to messengers ... + 'messenger'=>[ + 'slack'=>[ + 'presets'=>[<%= @replace["messenger-slack-presets"] %>], + ], + 'email'=>[ + 'from'=>[<%= @replace["messenger-email-from"] %>], + ], + ], + + 'foreman' => array( + 'api'=>'<%= @replace["foreman-url"] %>', // with ending "/" + 'user'=>'<%= @replace["foreman-user"] %>', + 'password'=>'<%= @replace["foreman-password"] %>', + 'ignore-ssl-error'=><%= @replace["foreman-ignore-ssl-error"] %>, + // 'varname-replace'=>'ci-replacement', + ), + + // ---------------------------------------------------------------------- + +]; \ No newline at end of file diff --git a/hooks/templates/inc_projects_config.php.erb b/hooks/templates/inc_projects_config.php.erb deleted file mode 100644 index 7dbb5f10259b923e73a574e8b486586c850e1624..0000000000000000000000000000000000000000 --- a/hooks/templates/inc_projects_config.php.erb +++ /dev/null @@ -1,202 +0,0 @@ -<?php - -// ---------------------------------------------------------------------- -// fetch status infos von den einzelnen Phasen -// ---------------------------------------------------------------------- - -$aConfig = array( - // Basispfad: - 'workDir' => '/var/imldeployment', - 'tmpDir' => '/var/tmp/imldeployment', - 'versionsToKeep' => 5, // for cleanup: keep n unused versions - 'builtsToKeep' => 3, - 'build' => array( - 'env' => 'export RVMSCRIPT="/usr/local/rvm/scripts/rvm";', - 'hooks' => array( - 'build-postclone' => 'hooks/onbuild-postclone', - 'build-precompress' => 'hooks/onbuild', - ), - ), - 'lang' => 'de', // for available languages see ./config/lang/*.json - - // rsync of archives - 'mirrorPackages' => array(), - - // task#4574 - // plugins - 'plugins'=>array( - - // enabled rollout plugins - 'rollout'=>array( - 'default'=>array(), - 'ssh'=>array( - 'user'=>'<%= @replace["rollout-ssh-user"] %>', - 'privatekey'=>'', - 'addkeycommand'=>'/usr/bin/ssh-keygen -R %s; /usr/bin/ssh-keyscan -t rsa %s >> /home/www-data/.ssh/known_hosts', - 'testcommand'=>'sudo puppet --version', - // 'command'=>'sudo puppet agent -t --detailed-exitcodes ; rc=$?; if [ $rc -eq 2 ]; then rc=0; fi ; exit $rc', - 'command'=>'/usr/local/bin/puppetrun.sh', - ), - 'awx'=>array( - 'url'=>'<%= @replace["rollout-awx-url"] %>', // no ending "/" - 'user'=>'<%= @replace["rollout-awx-user"] %>', - 'password'=>'<%= @replace["rollout-awx-password"] %>', - 'jobtemplate'=>'36', // 36 = iml wrapper playbook - 'tags'=>'rollout', - // 'ignore-ssl-error'=>false, - ), - /* - 'foreman'=>array( - 'api'=>'https://foreman.one.iml.unibe.ch/', // with ending "/" - 'user'=>'ci-server', - 'password'=>'A2h0MUVcnfRN_KLYMpymgjsHv0wu8qiY', - 'ignore-ssl-error'=>true, - ), - * - */ - ), - ), - 'installPackages' => array( - 'user' => 'imldeployment', - - // command to update ssh hostkey in known_hosts file - // %s is name of the server (2x) - 'addkeycommand' => '/usr/bin/ssh-keygen -R %s; /usr/bin/ssh-keyscan -t rsa %s >> /home/www-data/.ssh/known_hosts', - - // command to verify if puppet host is correct - 'testcommand' => 'sudo puppet --version', - - // puppet agent liefert 0 oder 2 zurueck, wenn OK - // http://docs.puppetlabs.com/references/3.4.0/man/apply.html - // 'command' => 'sudo puppet agent -t --detailed-exitcodes ; rc=$?; if [ $rc -eq 2 ]; then rc=0; fi ; exit $rc', - // task#38890 replace direct puppet call with a shell script - 'command' => '/usr/local/bin/puppetrun.sh', - ), - 'phases' => array( - "preview" => array( - 'css' => array( - 'bgdark' => 'background:#393E50; color:#f8f8f8;', - 'bglight' => 'background:#eee; color:#333; background:rgba(210,210,210,0.3); ', - 'bgbutton' => 'background:#393E50; color:#fcfcfc; border: 1px solid rgba(0,0,0,0.15);', - ), - ), - "stage" => array( - 'css' => array( - 'bgdark' => 'background:#3F88C5; color:#f8f8f8;', - 'bglight' => 'background:#f0f4f8; color:#333; background:rgba(200,210,220,0.3); ', - 'bgbutton' => 'background:#3F88C5; color:#fcfcfc; border: 1px solid rgba(0,0,0,0.15);', - ), - ), - "live" => array( - 'css' => array( - 'bgdark' => 'background:#44BBA4; color:#f8f8f8;', - 'bglight' => 'background:#f4f8f0; color:#333; background:rgba(210,220,200,0.3); ', - 'bgbutton' => 'background:#44BBA4; color:#fcfcfc; border: 1px solid rgba(0,0,0,0.15);', - ), - // wenn deploytimes existiert, dann wird nach dem Deploy das Paket - // in einer Queue zurueckgehalten - "deploytimes" => array('/(Mon|Tue|Wed|Thu)\ 14\:/'), - ), - ), - 'showdebug' => array( - 'ip'=>array(), - ), - - // generate template in hook/templates - 'auth' => array( - 'ldap' => array( - 'server' => '<%= @replace["ldap-url"] %>', - 'port' => 636, - 'DnLdapUser' => '<%= @replace["ldap-user"] %>', - 'PwLdapUser' => '<%= @replace["ldap-password"] %>', - 'DnUserNode' => '<%= @replace["ldap-dn-user"] %>', - 'DnAppNode' => '<%= @replace["ldap-cn-apps"] %><%= @replace["ldap-dn-apps"] %>', - 'debugLevel' => 0, - ) - ), - 'foreman' => array( - 'api'=>'<%= @replace["foreman-url"] %>', // with ending "/" - 'user'=>'<%= @replace["foreman-user"] %>', - 'password'=>'<%= @replace["foreman-password"] %>', - 'ignore-ssl-error'=><%= @replace["ignore-ssl-error"] %>, - // 'varname-replace'=>'ci-replacement', - ), - // where to store project data - 'projects' => array( - 'json' => array( - 'active' => true, - ), - 'ldap' => array( - 'active' => false, - ), - ), - // notifications to messengers ... - 'messenger'=>array(<%= @replace["messenger"] %>), -); - -// ---------------------------------------------------------------------- -// override for local development -// ---------------------------------------------------------------------- - -// unsafe ... -// make a check that fits your environemnt -// php_uname("n") can send a short hostname (without domain) -$bProd=!!strpos(__DIR__, '/ci.iml.unibe.ch/'); - -switch (php_uname("n")) { - case "USER": - case "AAE49": - case "dev.ci.iml.unibe.ch": - $aConfig['workDir'] = "D:\imldeployment"; - break; - - case "ci.iml.unibe.ch": - case "ci": - - if ($bProd){ - // synch der Pakete nur auf dem Livesystem - $aConfig['mirrorPackages'] = array( - 'puppet' => array( - 'type' => 'rsync', - 'runas' => '', // www-data, // nur fuer commandline - 'target' => 'ladmin@calcium.iml.unibe.ch:/share/imldeployment', - ), - 'puppet.one' => array( - 'type' => 'rsync', - 'runas' => '', // www-data, // nur fuer commandline - 'target' => 'copy-deployment@puppet.one.iml.unibe.ch:/var/shared/imldeployment', - ), - 'pkg-server' => array( - 'type' => 'rsync', - 'runas' => '', // www-data, // nur fuer commandline - 'target' => 'copy-deployment@software.shared.se.iml.unibe.ch:/var/shared/imldeployment', - ), - ); - } - break; - - default: - break; -} -if (!array_key_exists('tmpDir', $aConfig) || !$aConfig["tmpDir"]){ - $aConfig["tmpDir"] = (getenv("temp") ? getenv("temp") : "/var/tmp") . '/imldeployment'; -} - -// ---------------------------------------------------------------------- -// TODO: include custom settings that were saved in the GUI -// ---------------------------------------------------------------------- - - -// ---------------------------------------------------------------------- -// generate some vars -// ---------------------------------------------------------------------- - -$aConfig = array_merge($aConfig, array( - 'appRootDir' => dirname(dirname(__FILE__)), - 'configDir' => dirname(__FILE__), - 'dataDir' => $aConfig['workDir'] . '/data', // to write data: ssh keys, projects, database - 'buildDir' => $aConfig['workDir'] . '/build', - 'buildDefaultsDir' => $aConfig['workDir'] . '/defaults', - 'packageDir' => $aConfig['workDir'] . '/packages', - 'archiveDir' => $aConfig['workDir'] . '/packages/_files', - )); diff --git a/hooks/templates/inc_user2roles.php.erb b/hooks/templates/inc_user2roles.php.erb index a42b5862a2ae4f3c5682b3ac55453c1578a5fcd2..0ad69f461a3ccad8cbf480519b59a52472e569f0 100644 --- a/hooks/templates/inc_user2roles.php.erb +++ b/hooks/templates/inc_user2roles.php.erb @@ -1,15 +1,24 @@ <?php /* + * TARGET: config/inc_user2roles.php + * ---------------------------------------------------------------------- + * * IML DEPLOYMENT GUI - * USERS and assigned roles + * LIST OF ROLES AND ASSIGNED USERS * - * remark: Every user logged in is member of "authenticated" + * A user can be assigned to multiple groups + * + * see permissions per role in inc_roles.php + * + * Remarks: + * - Non authenticated users are member in "all" + * - Every user logged in is member of "authenticated" */ +return [ + // "authenticated" => [], + // "developer" => [], + // "projectmanager" => [], -return array( - // "authenticated" => array(), - // "developer" => array(), - // "projectmanager" => array(), - "admin" => array(<%= @replace["adminusers"] %>), + "admin" => [<%= @replace["adminusers"] %>], -); +]; diff --git a/public_html/check-config.php b/public_html/check-config.php index ee675a30a8447e936d93782a692152a61566129d..e66c1f4f3002eaad0ce37295cb508b7223f683a3 100644 --- a/public_html/check-config.php +++ b/public_html/check-config.php @@ -152,7 +152,7 @@ if (!isset($aConfig) || !is_array($aConfig)) { // ---------------------------------------------------------------------- echo '<h2>Check keys with directories</h2>'; foreach (array( - 'appRootDir', + // 'appRootDir', 'configDir', 'workDir', 'dataDir', @@ -180,14 +180,13 @@ if (!isset($aConfig) || !is_array($aConfig)) { foreach (array( 'auth', 'build', - 'installPackages', 'lang', 'phases', 'projects', ) as $sKey) { echo "Key [$sKey] "; if (!array_key_exists($sKey, $aConfig)) { - echo '<span class="error">failed</span> missing key [$sKey] in config<br>'; + echo '<span class="error">failed</span> missing key ['.$sKey.'] in config<br>'; $aErrors[] = "* missing key [$sKey] in config\n"; } else { echo '<span class="ok">OK</span> exists<br>'; @@ -222,6 +221,7 @@ checkCommands(array( 'which rsync'=>'sync packages to puppet master', 'which git'=>'connect to git repositories', )); + echo '<h3>tools for IML projects</h3>'; checkCommands(array( 'which uglifyjs'=>'compress js', diff --git a/public_html/deployment/classes/actionlog.class.php b/public_html/deployment/classes/actionlog.class.php index e09d1da94a77921fa207d134eb428f932c9d7630..0f07dffe765125ce85091a5bfcc91f50a1b5d7cd 100644 --- a/public_html/deployment/classes/actionlog.class.php +++ b/public_html/deployment/classes/actionlog.class.php @@ -45,7 +45,7 @@ class Actionlog { */ public function __construct($sProject = false) { global $aConfig; - if (!is_array($aConfig) || !array_key_exists("appRootDir", $aConfig)) { + if (!isset($aConfig["appRootDir"])) { die(__CLASS__ . "::".__FUNCTION__." ERROR: configuration with \$aConfig was not loaded."); } $this->_dbfile = $aConfig['dataDir'] . '/database/logs.db'; @@ -82,7 +82,7 @@ class Actionlog { */ private function _makeQuery($sSql) { // $this->_log(__FUNCTION__."($sSql)"); - // echo "<pre>$sSql</pre>"; + // echo "<pre>".htmlentities($sSql)."</pre>"; $db = new PDO("sqlite:" . $this->_dbfile); $result = $db->query($sSql); /* @@ -113,7 +113,7 @@ class Actionlog { '" . $this->_sUser . "', '" . $this->_sProject . "', '" . $sAction . "', - '" . $sMessage . "' + '" . str_replace("'", '"', $sMessage) . "' ); "; /* @@ -164,24 +164,24 @@ class Actionlog { $sWhere = false; $aWhere=array(); - if (array_key_exists("project", $aFilter) && $aFilter["project"]) { + if (isset($aFilter["project"]) && $aFilter["project"]) { $aWhere[]='`project`="' . $this->_filterAllowedChars($aFilter["project"], '[a-z0-9\-\_]') . '"'; } - if (array_key_exists("from", $aFilter) && $aFilter["from"]) { + if (isset($aFilter["from"]) && $aFilter["from"]) { $aWhere[]='`time`>="' . $this->_filterAllowedChars($aFilter["from"], '[0-9\-\ \:]') . '"'; } - if (array_key_exists("to", $aFilter) && $aFilter["to"]) { + if (isset($aFilter["to"]) && $aFilter["to"]) { $aWhere[]='`time`<="' . $this->_filterAllowedChars($aFilter["to"], '[0-9\-\ \:]') . '"'; } $sSql.=(count($aWhere) ? 'WHERE '. implode(' AND ', $aWhere) : ''); - if (array_key_exists("order", $aFilter) && $aFilter["order"]) { + if (isset($aFilter["order"]) && $aFilter["order"]) { $sSql.=' ORDER BY ' . $this->_filterAllowedChars($aFilter["order"], '[a-z\`0-9\,\ ]'); } else { $sSql.=' ORDER BY id DESC '; } - if (array_key_exists("limit", $aFilter) && $aFilter["limit"]) { + if (isset($aFilter["limit"]) && $aFilter["limit"]) { $sSql.=' LIMIT ' . $this->_filterAllowedChars($aFilter["limit"], '[0-9\,\ ]'); } @@ -293,13 +293,13 @@ class Actionlog { } else { // write filters as hidden fields - if (array_key_exists("project", $aFilter)){ + if (isset($aFilter["project"])){ $aForms["filter"]["form"]['selectproject'] = array( 'type' => 'hidden', 'value' => $aFilter["project"] ); } - if (array_key_exists("limit", $aFilter)){ + if (isset($aFilter["limit"])){ $aForms["filter"]["form"]['selectlimit'] = array( 'type' => 'hidden', 'value' => $aFilter["limit"] diff --git a/public_html/deployment/classes/build.interface.php b/public_html/deployment/classes/build.interface.php new file mode 100644 index 0000000000000000000000000000000000000000..9c9abf36cf15b5c69b65699c9d8829eaea719b4f --- /dev/null +++ b/public_html/deployment/classes/build.interface.php @@ -0,0 +1,23 @@ +<?php +/** + * INTERFACE for rollout plugins + * + * @author hahn + */ +interface iBuildplugin { + + /** + * get an array of commands to check requirements + * if the plugin is able to work + * @return array + */ + public function checkRequirements(); + + /** + * get an array with shell commands to execute + * @return array + */ + public function getBuildCommands(); + +} + diff --git a/public_html/deployment/classes/build_base.class.php b/public_html/deployment/classes/build_base.class.php new file mode 100644 index 0000000000000000000000000000000000000000..08832899a3ee5dec7478d60725040e40da08b703 --- /dev/null +++ b/public_html/deployment/classes/build_base.class.php @@ -0,0 +1,216 @@ +<?php +require_once 'build.interface.php'; + +/** + * build_base class that will be extended in a rollout plugin + * see deployment/plugins/build/* + * + * @author axel + */ +class build_base implements iBuildplugin{ + + protected $_sBuildDir = false; + protected $_sOutfile = false; + + protected $_sLang = "en-en"; + protected $_aLang = []; + + + // --------------------------------------------------------------- + // CONSTRUCTOR + // --------------------------------------------------------------- + + /** + * initialize build plugin + * @param array $aParams hash with those possible keys + * lang string language, i.e. 'de' + * phase string name of phase in a project + * globalcfg array given global config $aConfig + * projectcfg array project config to generate config + * for project and all phases + * @return boolean + */ + public function __construct($aParams) { + + // set current plugin id - taken from plugin directory name above + $oReflection=new ReflectionClass($this); + $this->_sPluginId=basename(dirname($oReflection->getFileName())); + + // ----- init language + if (isset($aParams['lang'])){ + $this->setLang($aParams['lang']); + } else { + $this->setLang(); + } + if (isset($aParams['workdir'])){ + $this->setWorkdir($aParams['workdir']); + } + if (isset($aParams['outfile'])){ + $this->setOutfile($aParams['outfile']); + } + + return true; + } + + // --------------------------------------------------------------- + // LANGUAGE TEXTS + // --------------------------------------------------------------- + + /** + * get a translated text from lang_XX.json in plugin dir; + * If the key is missed it returns "[KEY :: LANG]" + * + * @see setLang() + * @param string $sKey key to find in lang file + * @return string + */ + protected function _t($sKey){ + return (isset($this->_aLang[$sKey]) && $this->_aLang[$sKey]) + ? $this->_aLang[$sKey] + : "[ $sKey :: $this->_sLang ]" + ; + } + + /** + * set language for output of formdata and other texts. + * This method loads the language file into a hash. The output of + * translated texts can be done with $this->_t("your_key") + * + * @see _t() + * @param string $sLang language code, i.e. "de" + * @return boolean + */ + public function setLang($sLang=false){ + $this->_sLang=$sLang ? $sLang : $this->_sLang; + + $oReflection=new ReflectionClass($this); + $sFile=dirname($oReflection->getFileName()) . '/lang_'.$this->_sLang.'.json'; + $this->_aLang=(file_exists($sFile)) ? json_decode(file_get_contents($sFile), 1) : $this->_aLang; + return true; + } + // --------------------------------------------------------------- + // SETTER + // --------------------------------------------------------------- + + + /** + * set build dir with sources + * @param string $sBuildDir full path of the build directory + * @return array + */ + public function setWorkdir($sBuildDir){ + return $this->_sBuildDir=$sBuildDir ? $sBuildDir : $this->_sBuildDir; + } + + /** + * set outfile name + * @param string $sOutFilename filename for output (without extension) + * @return array + */ + public function setOutfile($sOutFilename){ + return $this->_sOutfile=$sOutFilename ? $sOutFilename : $this->_sOutfile; + } + + // --------------------------------------------------------------- + // GETTER + // --------------------------------------------------------------- + + /** + * check requirements if the plugin could work + * @return array + */ + public function checkRequirements() { + return [ + 'echo "ERROR: The method checkRequirements() was not implemented in the build plugin ['.$this->getId().']"', + 'exit 1' + ]; + } + + /** + * get an array with shell commands to execute + * @return array + */ + public function getBuildCommands(){ + return [ + 'echo "ERROR: The method getBuildCommamds() was not implemented in the build plugin ['.$this->getId().']"', + 'exit 1' + ]; + } + + /** + * get string with current ID + * @return string + */ + public function getId(){ + return $this->_sPluginId; + } + + /** + * get string with plugin name (taken from plugin language file) + * @return string + */ + public function getName(){ + return $this->_t('plugin_name'); + } + + /** + * get string with plugin description (taken from plugin language file) + * @return string + */ + public function getDescription(){ + return $this->_t('description'); + } + /** + * get array read from info.json + * @return type + */ + public function getPluginInfos(){ + + if ($this->_aPlugininfos){ + return $this->_aPlugininfos; + } + + $oReflection=new ReflectionClass($this); + $sFile=dirname($oReflection->getFileName()) . '/info.json'; + $this->_aPlugininfos= (file_exists($sFile)) + ? json_decode(file_get_contents($sFile), 1) + : array('error'=> 'unable to read info file ['.$sFile.'].') + ; + return $this->_aPlugininfos; + } + + /** + * get the file extension of created output file (from plugin info.json) + */ + public function getExtension(){ + $aInfos=$this->getPluginInfos(); + return isset($aInfos['extension']) ? '.'.$aInfos['extension'] : ''; + } + /** + * set outfile name including extension (from plugin metadata) + * @param string $sOutFilename filename for output (without extension) + * @return array + */ + public function getOutfile(){ + return $this->_sOutfile.$this->getExtension(); + } + /** + * set outfile name + * @param string $sOutFilename filename for output (without extension) + * @return array + */ + public function getBuildDir(){ + return $this->_sBuildDir; + } + + // ---------------------------------------------------------------------- + // INTERFACE :: RENDERER + // ---------------------------------------------------------------------- + public function renderPluginBox(){ + $sReturn=''; + $aInfos=$this->getPluginInfos(); + + return '<strong>'.$this->getName().'</strong> ('.$this->getId().')<br> + '.$this->getDescription(); + } +} diff --git a/public_html/deployment/classes/cache.class.php b/public_html/deployment/classes/cache.class.php index 471c578bb180b8bddd50944c9d45a52425940c96..7077f416eda033b727634065f24cfd019df5bf09 100644 --- a/public_html/deployment/classes/cache.class.php +++ b/public_html/deployment/classes/cache.class.php @@ -491,5 +491,3 @@ class AhCache { } // ---------------------------------------------------------------------- -?> - diff --git a/public_html/deployment/classes/config-replacement.class.php b/public_html/deployment/classes/config-replacement.class.php index e8f9a675216d7f2c54f58520f519a0272feb6494..93fb09a85b937b4526583168ff42687cdf247b2c 100644 --- a/public_html/deployment/classes/config-replacement.class.php +++ b/public_html/deployment/classes/config-replacement.class.php @@ -50,7 +50,7 @@ class configreplacement { $aBuildfiles=$this->_oProject->getBuildfilesByPlace($this->_sPhase, 'ready2install'); } - if (!$aBuildfiles || !array_key_exists('types', $aBuildfiles) || !array_key_exists('templates', $aBuildfiles['types'])){ + if (!isset($aBuildfiles['types']['templates'])){ return false; } foreach ($aBuildfiles['types']['templates'] as $sFile){ @@ -110,12 +110,12 @@ class configreplacement { } // abort if no foreman connection was configured - if (!array_key_exists('foreman', $aConfig)) { + if (!isset($aConfig['foreman'])) { return false; } // return already cached result - if (array_key_exists($sProject, $this->_aForemanReplacements)){ + if (isset($this->_aForemanReplacements[$sProject])){ return $this->_aForemanReplacements[$sProject]; } @@ -147,7 +147,7 @@ class configreplacement { // HACK: phases are part of the hostname .. but not "live" ... and special handling for demo $aPrjConfig=$this->_oProject->getConfig(); - $bIsDemo=(array_key_exists('fileprefix', $aPrjConfig) + $bIsDemo=(isset($aPrjConfig['fileprefix']) && !strpos($aPrjConfig['fileprefix'], 'demo')===false); $sPhase=$bIsDemo ? 'demo' : $this->_sPhase; foreach($aHosts as $aName){ @@ -216,14 +216,14 @@ class configreplacement { $aReturn=array(); foreach ($aProject as $sFile=>$aParams){ $aReturn[$sFile]=array(); - if (array_key_exists('target', $aParams)){ + if (isset($aParams['target'])){ $aReturn[$sFile]['target']=$aParams['target']; } - if (array_key_exists('replace', $aParams)){ + if (isset($aParams['replace'])){ $aReplace=json_decode($aParams['replace'], 1); $aReturn[$sFile]['replace_source']=$aReplace; foreach ($aReplace as $sVarname=>$value){ - if (is_array($value) && array_key_exists('_'.$this->_sPhase.'_', $value)){ + if (isset($value['_'.$this->_sPhase.'_'])){ $value=$value['_'.$this->_sPhase.'_']; } $aReturn[$sFile]['replace'][$sVarname]=$value; @@ -252,13 +252,10 @@ class configreplacement { */ private function _getForemanBaseUrl(){ global $aConfig; - if (!array_key_exists('foreman', $aConfig) - || !array_key_exists('api', $aConfig['foreman']) - || !$aConfig['foreman']['api'] - ){ - return false; - } - return $aConfig['foreman']['api']; + return (isset($aConfig['foreman']['api'])) + ? $aConfig['foreman']['api'] + : false + ; } /** @@ -272,7 +269,7 @@ class configreplacement { return false; } $aTmp=$this->_getForemanReplacement(); - if (!array_key_exists('hostsphase', $aTmp)){ + if (!isset($aTmp['hostsphase'])){ return false; } require_once 'htmlguielements.class.php'; diff --git a/public_html/deployment/classes/deploy-foreman.class.php b/public_html/deployment/classes/deploy-foreman.class.php deleted file mode 100644 index b99127d115155e3d06941b29762f9786ca57f49d..0000000000000000000000000000000000000000 --- a/public_html/deployment/classes/deploy-foreman.class.php +++ /dev/null @@ -1,490 +0,0 @@ -<?php -/** - * deployForeman - * - * foreman access to API - * - * @example - * in project class - * $oForeman=new deployForeman($this->_aConfig['foreman']); - * - * // enable debugging - * $oForeman->setDebug(1); - * - * // self check - * $oForeman->selfcheck(); die(__FUNCTION__); - * - * // read operating systems and get id and title only - * $aForemanHostgroups=$oForeman->read(array( - * 'request'=>array( - * array('operatingsystems'), - * ), - * 'response'=>array( - * 'id','title' - * ), - * )); - * - * // read details for operating systems #4 - * $aForemanHostgroups=$oForeman->read(array( - * 'request'=>array( - * array('operatingsystems', 4), - * ), - * )); - * - * - * $aOptions ... can contain optional subkeys - * - request - * [] list of array(keyword [,id]) - * - filter (array) - * - search (string) - * - page (string) - * - per_page (string) - * - response (array) - * - list of keys, i.e. array('id', 'title') - - * @author hahn - */ -class deployForeman { - - protected $_aCfg=array(); - protected $_bDebug = false; - - protected $_aAllowedUrls=array( - 'api/'=>array( - ''=>array(), - 'architectures'=>array(), - 'audits'=>array('methods'=>array('GET')), - 'auth_source_ldaps'=>array(), - 'bookmarks'=>array(), - 'common_parameters'=>array(), - 'compliance'=>array(), - 'compute_attributes'=>array(), - 'compute_profiles'=>array(), - 'compute_resources'=>array(), - 'config_groups'=>array(), - 'config_reports'=>array(), - 'config_templates'=>array(), - 'dashboard'=>array('methods'=>array('GET')), - 'domains'=>array(), - 'environments'=>array(), - 'fact_values'=>array(), - 'filters'=>array(), - 'hosts'=>array(), - 'hostgroups'=>array(), - 'job_invocations'=>array(), - 'job_templates'=>array(), - 'locations'=>array(), - 'mail_notifications'=>array(), - 'media'=>array(), - 'models'=>array(), - 'operatingsystems'=>array('methods'=>array('GET')), - 'orchestration'=>array(), - 'organizations'=>array(), - 'permissions'=>array(), - 'plugins'=>array(), - 'provisioning_templates'=>array(), - 'ptables'=>array(), - 'puppetclasses'=>array(), - 'realms'=>array(), - 'remote_execution_features'=>array(), - 'reports'=>array(), - 'roles'=>array(), - 'settings'=>array(), - 'smart_class_parameters'=>array(), - 'smart_proxies'=>array(), - 'smart_variables'=>array(), - 'statistics'=>array('methods'=>array('GET')), - 'status'=>array('methods'=>array('GET')), - 'subnets'=>array(), - 'template_combinations'=>array(), - 'template_kinds'=>array('methods'=>array('GET')), - 'templates'=>array(), - 'usergroups'=>array(), - 'users'=>array(), - // ... - ), - 'api/v2/'=>array( - 'discovered_hosts'=>array(), - 'discovery_rules'=>array(), - ), - 'foreman_tasks/api/'=>array( - 'recurring_logics'=>array(), - 'tasks'=>array(), - ), - ); - - - protected $_aRequest=array(); - - - // ---------------------------------------------------------------------- - // constructor - // ---------------------------------------------------------------------- - - - public function __construct($aCfg) { - if(!is_array($aCfg) || !count($aCfg) || !array_key_exists('api', $aCfg)){ - die("ERROR: class ".__CLASS__." must be initialized with an array containing api config for foreman."); - return false; - } - $this->_aCfg=$aCfg; - - return true; - } - - // ---------------------------------------------------------------------- - // private functions - // ---------------------------------------------------------------------- - /** - * add a log messsage - * @global object $oLog - * @param string $sMessage messeage text - * @param string $sLevel warnlevel of the given message - * @return bool - */ - protected function log($sMessage, $sLevel = "info") { - global $oCLog; - return $oCLog->add(basename(__FILE__) . " class " . __CLASS__ . " - " . $sMessage, $sLevel); - } - - /** - * search url prefix in $this->_aAllowedUrls by given key - * @param type $sFunction - * @return type - */ - protected function _guessPrefixUrl($sFunction=false){ - $sReturn=''; - if (!$sFunction){ - $sFunction=$this->_aRequest['request'][0][0]; - } - foreach($this->_aAllowedUrls as $sPrefix=>$aUrls){ - foreach(array_keys($aUrls) as $sKeyword){ - if ($sFunction==$sKeyword){ - $sReturn=$sPrefix; - break; - } - } - } - return $sReturn; - } - - /** - * generate an url for foreman API request based on option keys - * @return string - */ - protected function _generateUrl(){ - if(!array_key_exists('request', $this->_aRequest)){ - die('ERROR: missing key [request]'); - } - $sReturn=$this->_aCfg['api']; - - $sPrefix=$this->_guessPrefixUrl(); - $sReturn.=$sPrefix; - - foreach($this->_aRequest['request'] as $aReqItem){ - if (!array_key_exists($aReqItem[0], $this->_aAllowedUrls[$sPrefix])){ - echo 'WARNING: wrong item: [' . $aReqItem[0]."]<br>\n"; - } - $sReturn.=$aReqItem[0] ? $aReqItem[0].'/' : ''; - if(count($aReqItem)>1){ - $sReturn.=(int)$aReqItem[1].'/'; - } - } - return $sReturn; - } - - /** - * add parameter for search and paging in an foreman API URL - * - * @return string - */ - protected function _generateParams(){ - if (!array_key_exists('filter', $this->_aRequest)){ - return ''; - } - $sReturn='?'; - - // TODO: "sort by" and "sort order" ... need to be added here ... somehow - foreach(array('page', 'per_page', 'search') as $sParam){ - if (array_key_exists($sParam, $this->_aRequest['filter'])){ - $sReturn.='&'.$sParam.'='.urlencode($this->_aRequest['filter'][$sParam]); - } - } - return $sReturn; - } - - /** - * make an http get request and return the response body - * it is called by _makeRequest - * $aRequest contains subkeys - * - url - * - method; one of GET|POST|PUT|DELETE - * - postdata; for POST only - * - * @param array $aRequest arrayurl for Foreman API - * @return string - */ - protected function _httpCall($aRequest=false, $iTimeout = 5) { - if ($aRequest){ - $this->_aRequest=$aRequest; - } - $this->log(__FUNCTION__ . " start <pre>".print_r($this->_aRequest,1)."</pre>"); - if (!function_exists("curl_init")) { - die("ERROR: PHP CURL module is not installed."); - } - - $sApiUser=array_key_exists('user', $this->_aCfg) ? $this->_aCfg['user'] : false; - $sApiPassword=array_key_exists('password', $this->_aCfg) ? $this->_aCfg['password'] : false; - - $sFullUrl=$sApiUrl.$this->_aRequest['url']; - $this->log(__FUNCTION__ . " ".$this->_aRequest['method']." " . $this->_aRequest['url']); - $ch = curl_init($this->_aRequest['url']); - - curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $this->_aRequest['method']); - if ($this->_aRequest['method']==='POST'){ - curl_setopt($ch, CURLOPT_POSTFIELDS, $this->_aRequest['postdata']); - } - curl_setopt($ch, CURLOPT_HEADER, 1); - curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); - curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); - curl_setopt($ch, CURLOPT_TIMEOUT, $iTimeout); - curl_setopt($ch, CURLOPT_USERAGENT, 'IML Deployment GUI :: ' . __CLASS__); - if ($sApiUser){ - curl_setopt($ch, CURLOPT_USERPWD, $sApiUser.":".$sApiPassword); - } - - $res = curl_exec($ch); - $aReturn=array('info'=>curl_getinfo($ch), 'error'=>curl_error($ch)); - curl_close($ch); - $this->log(__FUNCTION__ . " status ".$aReturn['info']['http_code'].' '.$this->_aRequest['method']." $sFullUrl"); - - $sHeader=substr($res, 0, $aReturn['info']['header_size']); - $aReturn['header']=explode("\n", $sHeader); - $aReturn['body']=str_replace($sHeader, "", $res); - - return $aReturn; - } - - /** - * write debug infos if enabled - * @param string $sMessage - * @return boolean - */ - protected function _writeDebug($sMessage){ - if ($this->_bDebug){ - echo "DEBUG :: ".__CLASS__." :: $sMessage<br>\n"; - } - return true; - } - - // ---------------------------------------------------------------------- - // public functions :: low level - // ---------------------------------------------------------------------- - - /** - * make an http(s) request to foreman and scan result for http code and - * content in response json; method returns an array with subkeys - * - info: curl info array - * - error: curl error message - * - header: http response headers - * - body: http response body - * - _json: parsed json data from response body - * - _OK: flag if result is OK and complete - * - _status: info - * - * * $aRequest contains subkeys - * - function --> to extract method and generate url - * - method; one of GET|POST|PUT|DELETE - * - postdata; for POST only - - * @param array $aRequest arrayurl for Foreman API - * @return array - */ - public function makeRequest($aRequest=false) { - if ($aRequest){ - $this->_aRequest=$aRequest; - } - $sStatus='unknown'; - $bOk=false; - - - // prevent missing data because of paging - if ($this->_aRequest['method']==='GET' && !array_key_exists('per_page', $this->_aRequest['filter'])){ - $this->_aRequest['filter']['per_page']=1000; - } - // TODO check postdata - if ($this->_aRequest['method']==='POST' && (!array_key_exists('postdata',$this->_aRequest) || !count($this->_aRequest['postdata']))){ - die("ERROR in ".__CLASS__."::".__FUNCTION__.": missing data to make a POST request"); - } - - $this->_aRequest['url']=$this->_generateUrl().$this->_generateParams(); - - // ----- request - $this->_writeDebug(__FUNCTION__ . ' start request <pre>'.print_r($this->_aRequest,1).'</pre>'); - $aReturn=$this->_httpCall(); - - // ----- check result - // check status - $iStatuscode=$aReturn['info']['http_code']; - if ($iStatuscode===0){ - $sStatus='wrong host'; - } - if ($iStatuscode>=200 && $iStatuscode<400){ - $sStatus='OK'; - $bOk=true; - } - if ($iStatuscode>=400 && $iStatuscode<500){ - $sStatus='error'; - } - if ($iStatuscode===404){ - $sStatus='wrong url'; - } - - // check result json - if($bOk){ - $aJson=json_decode($aReturn['body'], 1); - if (is_array($aJson)){ - if (array_key_exists('total', $aJson) && $aJson['total'] > $aJson['per_page']){ - $bOk=false; - $sStatus='Http OK, but incomplete results (paging)'; - } - $aReturn['_json']=$aJson; - } else { - $bOk=false; - $sStatus='Http OK, but wrong response'; - } - } - $aReturn['_OK']=$bOk; - $aReturn['_status']=$sStatus; - $this->_writeDebug(__FUNCTION__ . ' result of request <pre>'.print_r($aReturn,1).'</pre>'); - - return $aReturn; - } - - /** - * filter output for the response based on values $this->_aRequest['response'] - * @param array $aData response of $this->makeRequest(); - * @return array - */ - protected function _filterOutput($aData){ - if(!array_key_exists('response', $this->_aRequest)){ - return $aData; - } - $aTmp=array_key_exists('results', $aData['_json']) ? $aData['_json']['results'] : array($aData['_json']); - if(!count($aTmp)){ - return array(); - } - $aReturn=array(); - foreach($aTmp as $aItem){ - $aReturn[] = array_filter($aItem, function($key) { - return array_search($key, $this->_aRequest['response']) !==false; - }, ARRAY_FILTER_USE_KEY - ); - } - return $aReturn; - } - - - // ---------------------------------------------------------------------- - // public foreman functions - // ---------------------------------------------------------------------- - - /** - * enable/ disable debugging - * @param boolean $bNewDebugFlag new value; true|false - * @return boolean - */ - public function setDebug($bNewDebugFlag){ - return $this->_bDebug=$bNewDebugFlag; - } - - /** - * check for missing config entries - * @return type - */ - public function selfcheck() { - $sOut=''; - $sWarning=''; - $sOut.="<h1>selfcheck</h1>"; - $aApi=$this->read(array('request'=>array(array('')))); - if($aApi['_OK']){ - foreach($aApi['_json']['links'] as $sKey=>$aCalls){ - $sOut.="<h2>$sKey</h2><ul>"; - foreach ($aCalls as $sLabel=>$sUrl){ - $sOut.="<li>$sLabel .. $sUrl "; - $aTmp=preg_split('#\/#', $sUrl); - $sDir2=count($aTmp)>2 ? $aTmp[2] : '??'; - $sDir3=count($aTmp)>3 ? $aTmp[3] : '??'; - $sOut.="..... " - . ($this->_guessPrefixUrl($sDir2).$this->_guessPrefixUrl($sDir3) - ?'<span style="background:#cfc">OK</span>' - :'<span style="background:#fcc">miss</span>' - ) . ' ' . $sDir2.', '.$sDir3 . "</li>\n"; - if (!($this->_guessPrefixUrl($sDir2).$this->_guessPrefixUrl($sDir3))){ - $sWarning.="<li>$sKey - $sLabel - $sUrl</li>"; - } - } - $sOut.="</ul>"; - } - } else { - $sOut.='ERROR: unable to connect to foreman or missing permissions.<br>'; - } - if ($sWarning){ - echo 'WARNINGS:<ol>'.$sWarning.'</ol>'; - } - echo $sOut; - return true; - } - - // ---------------------------------------------------------------------- - // public foreman API CRUD functions - // ---------------------------------------------------------------------- - - /** - * TODO: create - * @param array $aOptions - */ - public function create($aOptions){ - } - /** - * GETTER - * $aOptions ... can contain optional subkeys - * - request - * [] list of array(keyword [,id]) - * - filter (array) - * - search (string) - * - page (string) - * - per_page (string) - * - response (array) - * - list of keys, i.e. array('id', 'title') - * - * @param array $aOptions - * @return array - */ - public function read($aOptions){ - $this->_aRequest=$aOptions; - $this->_aRequest['method']='GET'; - $aData=$this->makeRequest(); - if (!$aData['_OK']){ - return false; - } - return $this->_filterOutput($aData); - } - - /** - * TODO - * @param type $aOptions - */ - public function update($aOptions){ - - } - - /** - * TODO - * @param type $aOptions - */ - public function delete($aOptions){ - - } - -} diff --git a/public_html/deployment/classes/foremanapi.class.php b/public_html/deployment/classes/foremanapi.class.php index e0e9851efb1eeb08f65c5b5dbabc01ee053ed24c..d6d8fd0648390bdcc8980084ec7dfd01a2975a24 100644 --- a/public_html/deployment/classes/foremanapi.class.php +++ b/public_html/deployment/classes/foremanapi.class.php @@ -133,7 +133,7 @@ class ForemanApi { public function __construct($aCfg) { - if(!is_array($aCfg) || !count($aCfg) || !array_key_exists('api', $aCfg)){ + if(!isset($aCfg['api'])){ die("ERROR: class ".__CLASS__." must be initialized with an array containing api config for foreman."); return false; } @@ -154,9 +154,7 @@ class ForemanApi { */ protected function log($sMessage, $sLevel = "info") { global $oCLog; - if($oCLog && method_exists($oLog, 'add')){ - return $oCLog->add(basename(__FILE__) . " class " . __CLASS__ . " - " . $sMessage, $sLevel); - } + return $oCLog->add(basename(__FILE__) . " class " . __CLASS__ . " - " . $sMessage, $sLevel); } /** @@ -188,7 +186,7 @@ class ForemanApi { * @return string */ protected function _generateUrl(){ - if(!array_key_exists('request', $this->_aRequest)){ + if(!isset($this->_aRequest['request'])){ die('ERROR: missing key [request]'); } $sReturn=$this->_aCfg['api']; @@ -197,7 +195,7 @@ class ForemanApi { $sReturn.=$sPrefix.'/'; foreach($this->_aRequest['request'] as $aReqItem){ - if (!array_key_exists($aReqItem[0], $this->_aAllowedUrls[$sPrefix])){ + if (!isset($this->_aAllowedUrls[$sPrefix][$aReqItem[0]])){ echo 'WARNING: wrong item: [' . $aReqItem[0]."]<br>\n"; } $sReturn.=$aReqItem[0] ? $aReqItem[0].'/' : ''; @@ -214,7 +212,7 @@ class ForemanApi { * @return string */ protected function _generateParams(){ - if (!array_key_exists('filter', $this->_aRequest)){ + if (!isset($this->_aRequest['filter'])){ return ''; } $sReturn='?'; @@ -246,10 +244,11 @@ class ForemanApi { die("ERROR: PHP CURL module is not installed."); } - $sApiUser=array_key_exists('user', $this->_aCfg) ? $this->_aCfg['user'] : false; - $sApiPassword=array_key_exists('password', $this->_aCfg) ? $this->_aCfg['password'] : false; + $sApiUser=isset($this->_aCfg['user']) ? $this->_aCfg['user'] : false; + $sApiPassword=isset($this->_aCfg['password']) ? $this->_aCfg['password'] : false; - $sFullUrl=$sApiUrl.$this->_aRequest['url']; + // $sFullUrl=$sApiUrl.$this->_aRequest['url']; + $sFullUrl=$this->_aRequest['url']; $this->log(__FUNCTION__ . " ".$this->_aRequest['method']." " . $this->_aRequest['url']); $ch = curl_init($this->_aRequest['url']); @@ -326,16 +325,23 @@ class ForemanApi { // prevent missing data because of paging - if ($this->_aRequest['method']==='GET' && !array_key_exists('per_page', $this->_aRequest['filter'])){ + if ($this->_aRequest['method']==='GET' && !isset($this->_aRequest['filter']['per_page'])){ $this->_aRequest['filter']['per_page']=1000; } // TODO check postdata - if ($this->_aRequest['method']==='POST' && (!array_key_exists('postdata',$this->_aRequest) || !count($this->_aRequest['postdata']))){ + if ($this->_aRequest['method']==='POST' && ( + !isset($this->_aRequest['postdata']) + || !is_array($this->_aRequest['postdata']) + || !count($this->_aRequest['postdata']) + ) + ){ die("ERROR in ".__CLASS__."::".__FUNCTION__.": missing data to make a POST request"); } - if (!array_key_exists('url', $this->_aRequest)){ - $this->_aRequest['url']=$this->_generateUrl().$this->_generateParams(); - } + + $this->_aRequest['url']=isset($this->_aRequest['url']) + ? $this->_aRequest['url'] + : $this->_generateUrl().$this->_generateParams() + ; // ----- request $this->_writeDebug(__FUNCTION__ . ' start request <pre>'.print_r($this->_aRequest,1).'</pre>'); @@ -362,7 +368,7 @@ class ForemanApi { if($bOk){ $aJson=json_decode($aReturn['body'], 1); if (is_array($aJson)){ - if (array_key_exists('total', $aJson) && $aJson['total'] > $aJson['per_page']){ + if (isset($aJson['total']) && $aJson['total'] > $aJson['per_page']){ $bOk=false; $sStatus='Http OK, but incomplete results (paging)'; } @@ -386,11 +392,10 @@ class ForemanApi { * @return array */ protected function _filterOutput($aData){ - if(!array_key_exists('response', $this->_aRequest)){ + if(!isset($this->_aRequest['response'])){ return $aData; } - $bIsList=array_key_exists('results', $aData['_json']); - $aTmp=$bIsList ? $aData['_json']['results'] : array($aData['_json']); + $aTmp=isset($aData['_json']['results']) ? $aData['_json']['results'] : array($aData['_json']); if(!count($aTmp)){ return array(); } @@ -404,9 +409,12 @@ class ForemanApi { } $aReturn[] = $aReturnitem; } + /* return ($bIsList==1) ? $aReturn : (count($aReturn) ? $aReturn[0] : array()); + */ + return $aReturn; } diff --git a/public_html/deployment/classes/formgen.class.php b/public_html/deployment/classes/formgen.class.php index 0e1ea574d1ed829579ef5bc0d279687aab368dc9..0cd6f23de7f33a546c8284d8028e8ec889069ff9 100644 --- a/public_html/deployment/classes/formgen.class.php +++ b/public_html/deployment/classes/formgen.class.php @@ -23,7 +23,7 @@ class formgen { * @return boolean */ public function __construct($aNewFormData = array()) { - if (count($aNewFormData)){ + if (is_array($aNewFormData) && count($aNewFormData)){ return $this->setFormarray($aNewFormData); } return true; @@ -48,14 +48,14 @@ class formgen { */ public function renderHtml($sFormId) { $sReturn = false; - if (!array_key_exists($sFormId, $this->aForm)) { + if (!isset($this->aForm[$sFormId])) { die("ERROR: " . __CLASS__ . ":" . __FUNCTION__ . " - form id " . $sFormId . " does not exist."); } // FORM tag $sReturn.='<form '; - if (array_key_exists("meta", $this->aForm[$sFormId])) { + if (isset($this->aForm[$sFormId]["meta"])) { foreach (array("method", "action", "target", "accept-charset", "class", "id", "name") as $sAttr) { - if (array_key_exists($sAttr, $this->aForm[$sFormId]["meta"])) { + if (isset($this->aForm[$sFormId]["meta"][$sAttr])) { $sReturn.=$sAttr . '="' . $this->aForm[$sFormId]["meta"][$sAttr] . '" '; } } @@ -80,10 +80,10 @@ class formgen { private function _addHtmlAtrributes($aAttributes, $elementData) { $sReturn = false; foreach ($aAttributes as $sAtrr) { - if (array_key_exists($sAtrr, $elementData) && $elementData[$sAtrr]) { - if ($sReturn) - $sReturn.=' '; - $sReturn.=$sAtrr . '="' . $elementData[$sAtrr] . '"'; + if (isset($elementData[$sAtrr]) && $elementData[$sAtrr]) { + $sReturn.=($sReturn ? ' ' : '') + .$sAtrr . '="' . $elementData[$sAtrr] . '"' + ; } } return $sReturn; @@ -102,7 +102,7 @@ class formgen { private function _checkReqiredKeys($aArray, $aRequiredKeys, $sLabel = false) { $bReturn = true; foreach ($aRequiredKeys as $sKey) { - if (!array_key_exists($sKey, $aArray)) { + if (!isset($aArray[$sKey])) { die("ERROR: $sLabel<br>Missing key \"$sKey\" in the array of a form element:<pre>" . print_r($aArray, true) . "</pre>"); $bReturn = false; } @@ -130,7 +130,7 @@ class formgen { . "title" ; - if (!array_key_exists("type", $elementData)) { + if (!isset($elementData["type"])) { print_r($elementData); die("ERROR: " . __CLASS__ . ":" . __FUNCTION__ . " - key "type" does not exist."); } @@ -142,9 +142,9 @@ class formgen { $sHtmlDefault = ''; $sHtmlTable = ''; - if (array_key_exists("label", $elementData)) { + if (isset($elementData["label"])) { $sLabelText = $elementData["label"]; - $sLabelText.=(array_key_exists("required", $elementData) && $elementData["required"]) ? $this->sRequired : ''; + $sLabelText.=(isset($elementData["required"]) && $elementData["required"]) ? $this->sRequired : ''; } switch ($elementData["type"]) { @@ -190,7 +190,7 @@ class formgen { break; case "markup": - if (array_key_exists("value", $elementData)) + if (isset($elementData["value"])) $sHtmlDefault = $elementData["value"] . "\n"; break; @@ -224,7 +224,7 @@ class formgen { case "select": // HINWEIS optgroups werden nicht unterstuezt - nur einfache Listen $this->_checkReqiredKeys($elementData, array("name")); - $sDivClass=(array_key_exists("inline", $elementData) && $elementData["inline"])?"form-group":"col-sm-10"; + $sDivClass=(isset($elementData["inline"]) && $elementData["inline"])?"form-group":"col-sm-10"; $sFormElement.='<div class="'.$sDivClass.'"><select id="' . $sId . '" class="form-control" '; $sFormElement.=$this->_addHtmlAtrributes(explode(",", "$sDefaultAttributes,name,onchange"), $elementData); $sFormElement.=">\n"; @@ -239,7 +239,7 @@ class formgen { if ($sLabelText) { // $sLabelElement.='<span class="help-block">' . $sLabelText . '</span>' . "\n"; - $sLabelClass=(array_key_exists("inline", $elementData) && $elementData["inline"])?"":"col-sm-2"; + $sLabelClass=(isset($elementData["inline"]) && $elementData["inline"])?"":"col-sm-2"; $sLabelElement = $this->_addLabel($sLabelText, $sId, $sLabelClass); } @@ -252,7 +252,7 @@ class formgen { case "submit": $this->_checkReqiredKeys($elementData, array("value")); $sClass="btn btn-primary "; - if (array_key_exists("class", $elementData)){ + if (isset($elementData["class"])){ $sClass.=$elementData["class"]; } $elementData["class"]=$sClass; @@ -276,7 +276,7 @@ class formgen { $sFormElement.=' />'; $sFormElement.="\n"; - if (array_key_exists("inline", $elementData) && $elementData["inline"]){ + if (isset($elementData["class"]) && $elementData["inline"]){ $sLabelElement = $this->_addLabel($sLabelText, $sId, "col-sm-2"); // $sHtmlDefault = $sLabelElement . "\n" . $sFormElement . "\n"; $sHtmlDefault = $sLabelElement . '<div class="col-sm-3">' . "\n" . $sFormElement . '</div>' . "\n"; @@ -313,11 +313,11 @@ class formgen { } // Default or table mode? - if (array_key_exists("mode", $elementData) && $elementData["mode"] == 'table' && $sHtmlTable) { + if (isset($elementData["mode"]) && $elementData["mode"] == 'table' && $sHtmlTable) { $sHtmlDefault = $sHtmlTable; } else { if ($elementData["type"] != "button" && $elementData["type"] != "fieldset" && $elementData["type"] != "markup") { - if (!array_key_exists("inline", $elementData) || !$elementData["inline"]){ + if (!isset($elementData["inline"]) || !$elementData["inline"]){ // $sHtmlDefault = "<fieldset>" . $sHtmlDefault . "</fieldset>\n"; $sHtmlDefault = '<div class="form-group">' . $sHtmlDefault . '</div>'."\n"; } diff --git a/public_html/deployment/classes/htmlguielements.class.php b/public_html/deployment/classes/htmlguielements.class.php index 7efad72e9dacde729c80dc8250176d80785aa913..4ebc040673ee3e1c0ff1848716f0b8ec34963939 100644 --- a/public_html/deployment/classes/htmlguielements.class.php +++ b/public_html/deployment/classes/htmlguielements.class.php @@ -220,7 +220,7 @@ class htmlguielements{ * @return string */ public function addAttributeFromKey($sAttribute, $aData, $sDefault=''){ - return (array_key_exists($sAttribute, $aData) + return (isset($aData[$sAttribute]) ? $this->addAttribute($sAttribute, $aData[$sAttribute]) : $this->addAttribute($sAttribute, $sDefault) ); @@ -281,7 +281,7 @@ class htmlguielements{ * @return array */ public function getIconByType($sType){ - return (array_key_exists($sType, $this->aCfg['icons']) + return (isset($this->aCfg['icons'][$sType]) ? $this->getIcon($this->aCfg['icons'][$sType]) : '' ); @@ -296,11 +296,11 @@ class htmlguielements{ public function getLink($aItem){ $sHref=$this->addAttributeFromKey('href', $aItem, '#'); - $sLabel=(array_key_exists('icon', $aItem) ? $this->getIcon($aItem['icon']): '') - .(array_key_exists('label', $aItem) ? $aItem['label'] : ''); + $sLabel=(isset($aItem['icon']) ? $this->getIcon($aItem['icon']): '') + .(isset($aItem['label']) ? $aItem['label'] : ''); foreach(array('href', 'icon', 'label') as $sKey){ - if (array_key_exists($sKey, $aItem)){ + if (isset($aItem[$sKey])){ unset($aItem[$sKey]); } } @@ -322,7 +322,7 @@ class htmlguielements{ */ protected function _getButtonattributesByType($aItem){ $aReturn=$aItem; - if (array_key_exists($aItem['type'], $this->aCfg['buttons'])){ + if (isset($this->aCfg['buttons'][$aItem['type']])){ $sClass=$this->aCfg['buttons'][$aItem['type']]['class']; $aReturn['class'].=$sClass ? ' '.$sClass : ''; @@ -331,7 +331,11 @@ class htmlguielements{ $aReturn['icon']=$aReturn['icon'] ? $aReturn['icon'] : ( $this->aCfg['buttons'][$aItem['type']]['icon'] ? $this->aCfg['buttons'][$aItem['type']]['icon'] - : ( array_key_exists($aItem['type'], $this->aCfg['icons']) ? $this->aCfg['icons'][$aItem['type']] : '') + : ( + isset($this->aCfg['icons'][$aItem['type']]) + ? $this->aCfg['icons'][$aItem['type']] + : '' + ) ); } return $aReturn; @@ -346,13 +350,13 @@ class htmlguielements{ */ public function getLinkButton($aItem){ foreach(array('class', 'icon') as $sKey){ - if (!array_key_exists($sKey, $aItem)){ + if (!isset($aItem[$sKey])){ $aItem[$sKey]=''; } } $aItem['class'].='btn btn-default'; - if (array_key_exists('type', $aItem)){ + if (isset($aItem['type'])){ $aItem=$this->_getButtonattributesByType($aItem); unset($aItem['type']); } @@ -379,7 +383,7 @@ class htmlguielements{ ); $sClass = ""; $sPrefix = ""; - if (array_key_exists($sWarnlevel, $aCfg)) { + if (isset($aCfg[$sWarnlevel])) { $sClass = $aCfg[$sWarnlevel]["class"]; $sPrefix = $aCfg[$sWarnlevel]["prefix"]; $sMessage = '<strong>' . $aCfg[$sWarnlevel]["prefix"] . '</strong> ' . $sMessage; @@ -406,10 +410,10 @@ class htmlguielements{ } $sNavType=$aTabData['options']['type']; // "tabs" or "pills" $sNavCss='nav nav-'.$sNavType; - if (array_key_exists('stacked', $aTabData['options']) && $aTabData['options']['stacked']){ + if (isset($aTabData['options']['stacked']) && $aTabData['options']['stacked']){ $sNavCss.=' nav-stacked'; } - if (array_key_exists('justified', $aTabData['options']) && $aTabData['options']['justified']){ + if (isset($aTabData['options']['justified']) && $aTabData['options']['justified']){ $sNavCss.=' nav-justified'; } $sNavType=$aTabData['options']['justified']; @@ -443,7 +447,7 @@ class htmlguielements{ public function getTable($aTabledata) { $sTHead=''; $sTBody=''; - if (array_key_exists('body', $aTabledata)){ + if (isset($aTabledata['body'])){ foreach ($aTabledata['body'] as $aRow){ $sTBody.='<tr>'; foreach ($aRow as $sItem){ @@ -452,7 +456,7 @@ class htmlguielements{ $sTBody.='</tr>'; } } - if (array_key_exists('header', $aTabledata)){ + if (isset($aTabledata['header'])){ foreach ($aTabledata['header'] as $sItem){ $sTHead.='<th>'.$sItem.'</th>'; } diff --git a/public_html/deployment/classes/ldap.class.php b/public_html/deployment/classes/ldap.class.php index d75e5971af70b0d0fae5d596a3cc3bba6f90c2a7..0a1a757635759ccde8e3361a616a2b75b2b699df 100644 --- a/public_html/deployment/classes/ldap.class.php +++ b/public_html/deployment/classes/ldap.class.php @@ -4,6 +4,7 @@ * IML LDAP CONNECTOR FOR USER AUTHENTICATION * * @author axel.hahn@iml.unibe.ch + * 07-2017 */ class imlldap { @@ -283,7 +284,7 @@ class imlldap { $aItems = $this->searchUser($sSearchFilter, $aAttributesToGet); - if(count($aItems)==2){ + if(is_array($aItems) && count($aItems)==2){ $this->_w(__FUNCTION__ . ' OK: I got a single result: ' . print_r($aItems[0],1) ); return $aItems[0]; } diff --git a/public_html/deployment/classes/logger.class.php b/public_html/deployment/classes/logger.class.php index 3f572ea0925af6279ebcc1b3e679ef4c25f79403..7da8dacbac4cf19a9870bb2b55f62ae4bb852673 100644 --- a/public_html/deployment/classes/logger.class.php +++ b/public_html/deployment/classes/logger.class.php @@ -8,6 +8,7 @@ class logger { protected $aMessages=array(); protected $bShowDebug=true; + protected $_iMemStart = false; /** @@ -16,6 +17,7 @@ class logger { * @return boolean */ public function __construct($sInitMessage="Logger was initialized.") { + $this->_iMemStart=memory_get_usage(); $this->add($sInitMessage); return true; } @@ -31,7 +33,7 @@ class logger { /** * enable client debugging by a given array of allowed ip addresses - * @param type $aIpArray + * @param array $aIpArray * @return boolean */ public function enableDebugByIp($aIpArray){ @@ -75,6 +77,12 @@ class logger { return false; } $sOut=''; + $iMem=memory_get_usage(); + $this->add('<hr>'); + $this->add('Memory on start: ' . number_format($this->_iMemStart, 0, '.', ',') . " bytes"); + $this->add('Memory on end: ' . number_format($iMem, 0, '.', ',') . " bytes"); + $this->add('Memory peak: ' . number_format(memory_get_peak_usage(), 0, '.', ',') . " bytes"); + $sStarttime=$this->aMessages[0]["time"]; $iCounter=0; diff --git a/public_html/deployment/classes/plugins.class.php b/public_html/deployment/classes/plugins.class.php new file mode 100644 index 0000000000000000000000000000000000000000..26c5869df5a812b25b8e3e5d0c2bf4a96944c54b --- /dev/null +++ b/public_html/deployment/classes/plugins.class.php @@ -0,0 +1,191 @@ +<?php + +/** + * WIP + * base class for all plugin types to read available plugins + * and its metadata + * + * @example + * $CI_plugins=new ciplugins(); + * print_r($CI_plugins->getPluginTypes()); + * + * // $CI_plugins->setType('build'); + * // print_r($CI_plugins->getPlugins()); + * print_r($CI_plugins->getPlugins('build')); + * + * $CI_plugins->setPlugin('tgz', 'build'); + * + * + * @author axel + */ +class ciplugins { + + protected $_sPlugindir=false; + + /** + * @var string + */ + protected $_sType=false; + + /** + * @var string + */ + protected $_sPluginname=false; + + + protected $_sLang = "en-en"; + protected $_aLang = []; + + + // --------------------------------------------------------------- + // CONSTRUCTOR + // --------------------------------------------------------------- + + /** + * initialize plugins + * @return boolean + */ + public function __construct() { + + $this->_sPlugindir=dirname(__DIR__).'/plugins'; + + return true; + } + + // --------------------------------------------------------------- + // LANGUAGE TEXTS + // --------------------------------------------------------------- + + /** + * get a translated text from lang_XX.json in plugin dir; + * If the key is missed it returns "[KEY :: LANG]" + * + * @see setLang() + * @param string $sKey key to find in lang file + * @return string + */ + protected function _t($sKey){ + return (isset($this->_aLang[$sKey]) && $this->_aLang[$sKey]) + ? $this->_aLang[$sKey] + : "[ $sKey :: $this->_sLang ]" + ; + } + + /** + * set language for output of formdata and other texts. + * This method loads the language file into a hash. The output of + * translated texts can be done with $this->_t("your_key") + * + * @see _t() + * @param string $sLang language code, i.e. "de" + * @return boolean + */ + public function setLang($sLang=false){ + $this->_sLang=$sLang ? $sLang : $this->_sLang; + + $oReflection=new ReflectionClass($this); + $sFile=dirname($oReflection->getFileName()) . '/lang_'.$this->_sLang.'.json'; + $this->_aLang=(file_exists($sFile)) ? json_decode(file_get_contents($sFile), 1) : $this->_aLang; + return true; + } + // --------------------------------------------------------------- + // SETTER + // --------------------------------------------------------------- + + /** + * set a type for plugins ... what is a name of a subdir in the plugins directory + * @param {string} $sType Name of a plugin type, e.g. build|rollout + */ + public function setType($sType){ + if(!$sType || !is_dir($this->_sPlugindir.'/'.$sType)){ + return false; + } + return $this->_sType=$sType; + } + + /** + * set a plugin with autoload + * + * @param {string} $sPluginName name of the plugin + * @param {string} $sType optuional: set a type + * @return bool + */ + public function setPlugin($sPluginName,$sType=false){ + if($sType){ + if (!$this->setType($sType)){ + return false; + } + } + $sFile=$this->getPluginFilename($sPluginName); + if(!file_exists($sFile)){ + return false; + } + include_once $sFile; + return $this->_sPluginname=$sPluginName; + } + + // --------------------------------------------------------------- + // GETTER + // --------------------------------------------------------------- + + + /** + * get a location of a plugin file with full path + * The type must be initialized first with setType() + * + * @param string $sPluginName optional: Name of plugin + * @return string + */ + public function getPluginFilename($sPluginName=false){ + if(!$sPluginName){ + $sPluginName=$this->_sPluginname; + } + return $this->_sPlugindir.'/'.$this->_sType.'/'.$sPluginName.'/'.$this->_sType.'_'.$sPluginName.'.php'; + } + + + /** + * get an array of available plugin types read from filesystem + * @return array + */ + public function getPluginTypes(){ + $aReturn=[]; + foreach(glob($this->_sPlugindir.'/*', GLOB_ONLYDIR) as $sMydir){ + $aReturn[]=basename($sMydir); + } + return $aReturn; + } + + /** + * get an array of available plugins read from filesystem + * @return array + */ + public function getPlugins($sType=false){ + $aReturn=[]; + if($sType){ + if (!$this->setType($sType)){ + return $aReturn; + } + } + foreach(glob($this->_sPlugindir.'/'.$this->_sType.'/*', GLOB_ONLYDIR) as $sMydir){ + $aReturn[]=basename($sMydir); + } + return $aReturn; + } + + /** + * get a location of a plugin file with full path + * @param {bool} $bAutoload flag: autoload needed plugin file + * @return string + */ + public function getPluginClassname(){ + return $this->_sType.'_'.$this->_sPluginname; + } + + public function initPlugin(){ + $sClassname=$this->_sType.'_'.$this->_sPlugindir; + $TmpRolloutPlugin = new $sClassname([]); + + } + +} diff --git a/public_html/deployment/classes/project.class.php b/public_html/deployment/classes/project.class.php index 941ddc370d4aa11fabdd364980a6f3e211b1937c..596e776addc128ea2f0669dc30531b623759ce55 100644 --- a/public_html/deployment/classes/project.class.php +++ b/public_html/deployment/classes/project.class.php @@ -7,6 +7,10 @@ require_once __DIR__.'/../inc_functions.php'; require_once 'base.class.php'; require_once 'htmlguielements.class.php'; require_once 'messenger.class.php'; + +// plugins +// require_once 'plugins.class.php'; +require_once 'build_base.class.php'; require_once 'rollout_base.class.php'; /* ###################################################################### @@ -208,7 +212,7 @@ class project extends base { * @return boolean */ private function _verifyConfig() { - if (!count($this->_aPrjConfig)){ + if (!is_array($this->_aPrjConfig) || !count($this->_aPrjConfig)){ // die(t("class-project-error-no-config")); throw new Exception(t("class-project-error-no-config")); } @@ -407,8 +411,9 @@ class project extends base { mkdir($this->_aConfig['packageDir'] . "/" . $sPhase); } - if ($sPlace == "onhold") + if ($sPlace == "onhold"){ $sBase.="_onhold"; + } // $sBase .= "/" . $this->_aPrjConfig["fileprefix"]; // url for deployed if ($sPlace == "deployed") { @@ -420,6 +425,7 @@ class project extends base { } } + // echo "DEBUG: ".__METHOD__."($sPhase, $sPlace) --> $sBase<br>"; return $sBase; } @@ -435,14 +441,14 @@ class project extends base { } /** - * get filename for package file (.tgz file) + * get filename for package file (without file extension) * @param string $sPhase one of preview|stage|live ... * @param string $sPlace one of onhold|ready2install|deployed * @return string */ private function _getPackagefile($sPhase, $sPlace) { $sBase = $this->_getFileBase($sPhase, $sPlace); - return $sBase ? $sBase . '/' . $this->_aPrjConfig["fileprefix"] . '.tgz' : false; + return $sBase ? $sBase . '/' . $this->_aPrjConfig["fileprefix"] : false; } /** @@ -474,6 +480,7 @@ class project extends base { $sIcon = 'file-template'; break; case 'tgz': + case 'zip': $sType = 'package'; $sIcon = 'file-archive'; break; @@ -522,6 +529,22 @@ class project extends base { return $this->_getBuildfilesByDir($this->_getProjectArchiveDir() . '/' . $sVersion); } + /** + * get the group id of the project + * @return string + */ + public function getProjectGroup() { + return isset($this->_aPrjConfig["projectgroup"]) && $this->_aPrjConfig["projectgroup"]!='-1' ? $this->_aPrjConfig["projectgroup"] : false; + } + /** + * get the group label of the project + * @return string + */ + public function getProjectGroupLabel() { + $sGroupid=$this->getProjectGroup(); + return isset($this->_aConfig["projectgroups"][$sGroupid]) ? $this->_aConfig["projectgroups"][$sGroupid] : false; + } + /** * get full path of a packed project archive * @param string $sVersion version number of the build @@ -865,7 +888,9 @@ class project extends base { } /** * get deploy and queue infos for all phases - * @return type + * It build up a subkey "progress" with info if a build is queued + * or an installation of a new package is going on + * @return array */ public function getAllPhaseInfos() { @@ -888,8 +913,9 @@ class project extends base { $aDataPhase = $this->_aData["phases"][$sPhase]; foreach (array_keys($this->getPlaces()) as $sPlace) { if ( - array_key_exists($sPlace, $aDataPhase) - && array_key_exists('version', $aDataPhase[$sPlace]) + $sPlace!=='onhold' + && array_key_exists($sPlace, $aDataPhase) + // && array_key_exists('version', $aDataPhase[$sPlace]) ) { if($bFirstVersion && !$bHasDifferentVersions && $bFirstVersion!==$aDataPhase[$sPlace]['version']){ $bHasDifferentVersions=true; @@ -952,20 +978,20 @@ class project extends base { $sJsonfile = $this->_getInfofile($sPhase, $sKey); $aTmp[$sKey] = array(); if (file_exists($sJsonfile)) { - $sPkgfile = $this->_getPackagefile($sPhase, $sKey); - if (file_exists($sPkgfile)) { + // $sPkgfile = $this->_getPackagefile($sPhase, $sKey); + // if (file_exists($sPkgfile)) { $aJson = json_decode(file_get_contents($sJsonfile), true); if (is_array($aJson) && array_key_exists("version", $aJson)) { $aTmp[$sKey] = $aJson; $aTmp[$sKey]["infofile"] = $sJsonfile; - $aTmp[$sKey]["packagefile"] = $sPkgfile; + // $aTmp[$sKey]["packagefile"] = $sPkgfile; $aTmp[$sKey]["ok"] = 1; } else { $aTmp[$sKey]["error"] = sprintf(t("class-project-error-metafile-has-no-version"), $sJsonfile, print_r($aJson, true)); } - } else { - $aTmp[$sKey]["error"] = sprintf(t("class-project-error-getPhaseInfos-package-not-found"), $sPkgfile); - } + // } else { + // $aTmp[$sKey]["error"] = sprintf(t("class-project-error-getPhaseInfos-package-not-found"), $sPkgfile); + // } } else { $aTmp[$sKey]["error"] = sprintf(t("class-project-error-metafile-does-not-exist"), $sJsonfile); } @@ -1019,6 +1045,7 @@ class project extends base { $this->_aData["phases"][$sPhase] = $aTmp; } + // echo '<pre>'.print_r($this->_aData["phases"][$sPhase], 1).'</pre>'.__METHOD__.'<br>'; return $this->_aData["phases"][$sPhase]; } @@ -1047,13 +1074,15 @@ class project extends base { */ public function isActivePhase($sPhase) { return ( - array_key_exists("active", $this->_aPrjConfig["phases"][$sPhase]) ? $this->_aPrjConfig["phases"][$sPhase]["active"][0] : false + $this->_aPrjConfig && isset($this->_aPrjConfig["phases"][$sPhase]["active"][0]) + ? $this->_aPrjConfig["phases"][$sPhase]["active"][0] + : false ); } /** * return array of all (active and inactive) phases - * @return type + * @return array */ public function getPhases() { return $this->_aConfig["phases"]; @@ -1127,7 +1156,7 @@ class project extends base { if (!$this->oUser->hasPermission("project-action-accept") && !$this->oUser->hasPermission("project-action-accept-$sPhase") ) { // echo $this->oUser->showDenied(); - return false; + return '<span class="btn" title="no permission [project-action-accept] for user [' . $this->oUser->getUsername() . ']">' . $sPhase . '</span>'; } if (!$sPhase) { @@ -1339,7 +1368,7 @@ class project extends base { if(!$sSection){ $aReturn=$this->_aConfig["plugins"]; } else { - foreach ($this->_aConfig["plugins"]["rollout"] as $sPluginName=>$aItem) { + foreach ($this->_aConfig["plugins"][$sSection] as $sPluginName=>$aItem) { $aReturn[$sPluginName] = $aItem; } } @@ -1495,7 +1524,7 @@ class project extends base { $this->oRolloutPlugin = new $sPluginClassname(array( 'lang'=>$this->_aConfig['lang'], 'phase'=>false, - 'globalcfg'=>$this->_aConfig['plugins']['rollout'][$sPluginName], + 'globalcfg'=>isset($this->_aConfig['plugins']['rollout'][$sPluginName]) ? $this->_aConfig['plugins']['rollout'][$sPluginName] : [], 'projectcfg'=>$this->_aPrjConfig, )); // print_r($this->_oRolloutPlugin->getPluginfos()); @@ -1895,11 +1924,42 @@ class project extends base { } $this->_TempFill($sReturn, $aActionList); - - // create tgz archive - $sReturn.=sprintf(t("creating-file"), $sPackageFileArchiv) . "<br>"; - $sReturn.=$this->_execAndSend("cd $sTempBuildDir && tar -czf $sPackageFileArchiv ."); - $this->_TempFill($sReturn, $aActionList); + // ----- loop over enabled build plugins + // WIP + // set name of the activated plugin for this project + $aPlugins=(isset($this->_aPrjConfig['build']['enabled_build_plugins']) && $this->_aPrjConfig['build']['enabled_build_plugins']) + ? $this->_aPrjConfig['build']['enabled_build_plugins'] + : ['tgz'] + ; + foreach($aPlugins as $sPluginName){ + $oPlugin = false; + $sReturn.='<h4>'.$sPluginName.'</h4>'; + try{ + include_once $this->_getPluginFilename('build', $sPluginName); + $sPluginClassname='build_'.$sPluginName; + $oPlugin = new $sPluginClassname(array( + 'lang'=>$this->_aConfig['lang'], + 'workdir'=>$sTempBuildDir, + 'outfile'=>$sPackageFileArchiv, + )); + } catch (Exception $ex) { + return $this->_oHtml->getBox("error", + "FAILED to initialize build plugin " .$sPluginName .'<br>' + . $sReturn + ); + } + + $sReturn.=sprintf(t("creating-file"), $oPlugin->getOutfile()) . "<br>"; + foreach($oPlugin->checkRequirements() as $sCommand){ + $sReturn.=$this->_execAndSend($sCommand); + $this->_TempFill($sReturn, $aActionList); + } + + foreach($oPlugin->getBuildCommands() as $sCommand){ + $sReturn.=$this->_execAndSend($sCommand); + $this->_TempFill($sReturn, $aActionList); + } + } // write info file (.json) $sReturn.=sprintf(t("creating-file"), $sInfoFileArchiv) . "<br>"; @@ -2200,7 +2260,7 @@ class project extends base { $sReturn.=t("class-project-info-deploy-start-by-method-skip") . "<br>"; } else { - $sReturn.='<p>' . 'Plugin: '.$this->oRolloutPlugin->getId().'</p>'; + $sReturn.='<p>Plugin: '.$this->oRolloutPlugin->getId().'</p>'; foreach($this->oRolloutPlugin->getDeployCommands($sPhase) as $sCmd){ $sReturn.=$this->_execAndSend("$sCmd"); @@ -2789,7 +2849,7 @@ class project extends base { } if (!$this->oUser->hasPermission("project-action-$sFunction")) { // $sClass .= ' disabled'; - return '<span title="no permission [project-action-' . $sFunction . '] for ' . $this->oUser->getUsername() . '">[ <i class="' . $sIconClass . '"></i> ' . $sLabel . ' ]</span>'; + return '<span class="btn disabled btn-default" title="no permission [project-action-' . $sFunction . '] for user [' . $this->oUser->getUsername() . ']"><i class="' . $sIconClass . '"></i> ' . $sLabel . '</span>'; } return $this->_oHtml->getLinkButton(array( @@ -3073,7 +3133,8 @@ class project extends base { $bIsError = false; $this->_oHtml = new htmlguielements(); - $sInfos.=''; + $sInfos=''; + $sTitle=''; if (array_key_exists("title", $aOptions) && $aOptions["title"]) { $sTitle.=$aOptions["title"]; } @@ -3208,14 +3269,22 @@ class project extends base { $sReturn = ''; $sContinue = '<span style="font-size: 300%; color:#ace;">»»</span><br><br>'; + $aBranches=$this->getRemoteBranches(); + if(!is_array($aBranches)){ + return t("project-setup-incomplete"); + } $sRepoBar = ''; + /* + Speedup: + $aRepodata = $this->getRepoRevision(); if (array_key_exists("revision", $aRepodata)) { $sRepoBar = $this->_getChecksumDiv($aRepodata["revision"]); } else { $sRepoBar = '<span class="error">' . t("error") . '</span>'; } + */ $sPackagebar = ''; $aVersions = $this->_getVersionUsage(); @@ -3249,57 +3318,56 @@ class project extends base { <div><img src="/deployment/images/process/bg_phase.png" alt="' . t("phase") . ' ' . $sPhase . '"></div> </div>'; } - $sReturn = ' - <div class="visualprocess"> - <div class="process box"> - <div class="title">' . $this->_oHtml->getIcon('repository') . t("versioncontrol") . '</div> - <div class="details"> - ' . $sRepoBar . '<br> - <!-- - <a href="#h3repo" class="scroll-link">' . t("repositoryinfos") . '</a><br> - --> - ' . t("repositoryinfos") . '<br> - <strong> - ' . $this->_aPrjConfig["build"]["type"] . '</strong> ' . preg_replace('/.*\@(.*):.*/', '($1)', $this->_aPrjConfig["build"]["url"]) - . ': <strong title="' . t('branch-select') . '">' . count($this->getRemoteBranches()) . '</strong>' - . '<br> + $sReturn = ' + <div class="visualprocess"> + <div class="process box"> + <div class="title">' . $this->_oHtml->getIcon('repository') . t("versioncontrol") . '</div> + <div class="details"> + ' . $sRepoBar . '<br> + <!-- + <a href="#h3repo" class="scroll-link">' . t("repositoryinfos") . '</a><br> + --> + ' . t("repositoryinfos") . '<br> + ' . $this->_aPrjConfig["build"]["type"] . '</strong> ' . preg_replace('/.*\@(.*):.*/', '($1)', $this->_aPrjConfig["build"]["url"]) + . ': <strong title="' . t('branch-select') . '">' . count($aBranches) . '</strong>' + . '<br> + </div> + <div> + <img src="/deployment/images/process/bg_vcs.png" alt="' . t("versioncontrol") . '"> + </div> </div> - <div> - <img src="/deployment/images/process/bg_vcs.png" alt="' . t("versioncontrol") . '"> + + <div class="process"> + <div class="title"> </div> + <div class="action">' . $sContinue . t("build-hint-overview") . '<br><br>' . ($this->canAcceptPhase() ? $this->renderLink("build") : '') . '</div> </div> - </div> - - <div class="process"> - <div class="title"> </div> - <div class="action">' . $sContinue . t("build-hint-overview") . '<br><br>' . ($this->canAcceptPhase() ? $this->renderLink("build") : '') . '</div> - </div> - - <div class="process box"> - <div class="title">' . $this->_oHtml->getIcon('package') . t("archive") . '</div> - <div class="details"> - ' . $sPackagebar . '<br> - <!-- - <a href="#h3versions" class="scroll-link">' . t("packages") . '</a><br> - --> - ' . t("packages") . '<br> - (<strong>' . count($this->_getVersionUsage()) . '</strong>) + + <div class="process box"> + <div class="title">' . $this->_oHtml->getIcon('package') . t("archive") . '</div> + <div class="details"> + ' . $sPackagebar . '<br> + <!-- + <a href="#h3versions" class="scroll-link">' . t("packages") . '</a><br> + --> + ' . t("packages") . '<br> + (<strong>' . count($this->_getVersionUsage()) . '</strong>) + </div> + <div><img src="/deployment/images/process/bg_archive.png" alt="' . t("archive") . '"></div> + </div> + + <div class="process"> + <div class="title"> </div> + <div class="action">'.$sContinue . sprintf(t("queue-hint-overview"), $this->getNextPhase()).'</div> + </div> + + <div class="process phases box"> + <div class="title">' . $this->_oHtml->getIcon('phase') . t("phases") . '</div> + ' . ($sPhaseImg ? $sPhaseImg : '<div class="process">' . t("none") . '</div>') . ' </div> - <div><img src="/deployment/images/process/bg_archive.png" alt="' . t("archive") . '"></div> - </div> - - <div class="process"> - <div class="title"> </div> - <div class="action">'.$sContinue . sprintf(t("queue-hint-overview"), $this->getNextPhase()).'</div> - </div> - - <div class="process phases box"> - <div class="title">' . $this->_oHtml->getIcon('phase') . t("phases") . '</div> - ' . ($sPhaseImg ? $sPhaseImg : '<div class="process">' . t("none") . '</div>') . ' </div> - </div> - '; - + '; + return $sReturn; } @@ -3314,17 +3382,36 @@ class project extends base { $sMessages = ''; require_once ("formgen.class.php"); - + + $aSelectProjectGroup = array( + 'type' => 'select', + 'name' => 'projectgroup', + 'label' => t("projectgroup"), + 'options' => array( + OPTION_NONE => array( + 'label' => t('none'), + ), + '' => array( + 'label' => '- - - - - - - - - - - - - - - - - - - - ', + ), + ), + ); + foreach($this->_aConfig['projectgroups'] as $sGroupid=>$sGroupLabel){ + $bActive=$this->getProjectGroup() === $sGroupid; + $aSelectProjectGroup['options'][$sGroupid] = array( + 'label' => $sGroupLabel, + 'selected' => $bActive ? 'selected' : false, + ); + } + $aSelectSlack = array( 'type' => 'hidden', 'name' => 'messenger[slack]', 'value' => false, ); if ( - array_key_exists('messenger', $this->_aConfig) - && array_key_exists('slack', $this->_aConfig['messenger']) - && array_key_exists('presets', $this->_aConfig['messenger']['slack']) - && count(array_key_exists('presets', $this->_aConfig['messenger']['slack']['presets'])) + isset($this->_aConfig['messenger']['slack']['presets']) + && count($this->_aConfig['messenger']['slack']['presets']) ) { $aSelectSlack = array( 'type' => 'select', @@ -3348,6 +3435,50 @@ class project extends base { } } + // ---------- Build plugins + /* + + $aPluginsBuild = array( + 'select' => array( + 'type' => 'checkbox', + 'name' => 'build[enabled_build_plugins]', + 'label' => t("build-plugins"), + 'options' => [], + ), + // 'project-config' => '', + ); + foreach (array_keys($this->getConfiguredPlugins('build')) as $sPluginName){ + + $sPluginFile=$this->_getPluginFilename('build', $sPluginName); + $TmpRolloutPlugin = false; + $sMyClassname='build_'. $sPluginName; + if(file_exists($sPluginFile)){ + try{ + include_once $this->_getPluginFilename('build', $sPluginName); + $TmpRolloutPlugin = new $sMyClassname([]); + echo "FOUND $sMyClassname<br>"; + $aPluginsBuild['select']['options'][$sPluginName]=array( + 'label' => $TmpRolloutPlugin->getName(), + 'checked' => $bActive, + // 'onclick' => '$(\'.'.$sMyDivClass.'\').hide(); $(\'.' . $sMyDivClassActive . '\').show();', + ); + } catch (Exception $ex) { + + } + } else { + $aRollout['project-select']['options'][$sPluginName]=array( + 'label' => 'not found: <span class="error">' . $sMyClassname . '</span>', + 'checked' => false, + 'disabled' => "disabled", + ); + + + } + } + echo '<pre>'; print_r($aPluginsBuild); die(__METHOD__); + */ + + // ---------- /Build plugins // ---------- Rollout plugins $aRollout = array( @@ -3449,7 +3580,7 @@ class project extends base { ), ), ); - if (count($aForemanHostgroups)) { + if ($aForemanHostgroups && count($aForemanHostgroups)) { foreach ($aForemanHostgroups as $aItem) { $bActive=$iForemanHostgroupDefault === (int) $aItem['id']; $aSelectForemanGroups['options'][$aItem['id']] = array( @@ -3569,7 +3700,9 @@ class project extends base { 'size' => 100, 'placeholder' => '', ), - + + 'input' . $i++ => $aSelectProjectGroup, + 'input' . $i++ => array( 'type' => 'markup', 'value' => '<p>' . t('messenger') . '</p>', @@ -3789,7 +3922,7 @@ class project extends base { ), ), ); - if (count($aForemanHostgroups)) { + if (is_array($aForemanHostgroups) && count($aForemanHostgroups)) { foreach ($aForemanHostgroups as $aItem) { $aSelectForemanHostGroup['options'][$aItem['id']] = array( 'label' => $aItem['title'], @@ -3942,7 +4075,7 @@ class project extends base { // 'required' => 'required', 'validate' => 'isastring', 'size' => 100, - 'placeholder' => implode(", ", $this->_aConfig["phases"][$sPhase]["deploytimes"]), + 'placeholder' => isset($this->_aConfig["phases"][$sPhase]["deploytimes"]) ? implode(", ", $this->_aConfig["phases"][$sPhase]["deploytimes"]) : '', ); $aForms["setup"]["form"]['input' . $i++] = array( 'type' => 'markup', diff --git a/public_html/deployment/classes/projectlist.class.php b/public_html/deployment/classes/projectlist.class.php index f750e99c9c5571fafb1698c037bb68a06b82ddb7..7ecdcc46eb69bf2fb988704ab47f539f12775d04 100644 --- a/public_html/deployment/classes/projectlist.class.php +++ b/public_html/deployment/classes/projectlist.class.php @@ -60,13 +60,16 @@ class projectlist extends base{ $oHtml=new htmlguielements(); $sPrjFilter = ''; + $sPrjGroupFilter = ''; $sPhaseFilter = ''; $sPrjFilter.='<option value="">' . t("all") . '</option>'; + $sPrjGroupFilter.='<option value="">' . t("all") . '</option>'; + $sPhaseFilter.='<option value="' . $sColClass . '">' . t("all") . '</option>'; $iInprogress=0; $iHasqueue=0; - + $aPrjGroups=[]; $sDivInprogress='<div class="progressinprogress" title="'.t("progress-inprogress").'">'.$oHtml->getIcon('refresh').t("progress-inprogress").'</div>'; $sDivHasqueue='<div class="progresshasqueue" title="'.t("progress-hasqueue").'">'.$oHtml->getIcon('waiting').t("progress-hasqueue").'</div>'; @@ -92,10 +95,13 @@ class projectlist extends base{ } } $aProgress=$oPrj->getProgress(); + $sPrjGroup=$oPrj->getProjectGroup(); + $sClasses=$sPrj . ' ' . $sTrClass . ' trprogress' .($aProgress['inprogress'] ? ' progressinprogress' : '') .($aProgress['hasQueue'] ? ' progresshasqueue' : '') + . ' group-'.$sPrjGroup ; if($aProgress['inprogress']){ @@ -107,6 +113,17 @@ class projectlist extends base{ $iHasqueue++; $sProgress.=$sDivHasqueue; } + + + if($sPrjGroup){ + if(!isset($aPrjGroups[$sPrjGroup])){ + $aPrjGroups[$sPrjGroup]=0; + $sPrjGroupFilter.='<option value="' . $sPrjGroup . '">' . $oPrj->getProjectGroupLabel() . '</option>'; + } else { + $aPrjGroups[$sPrjGroup]++; + } + } + $sOut2 .= '<div class="' . $sClasses . ' prjbox"><div class="title">' .$oHtml->getLink(array( 'href'=>'#', @@ -192,6 +209,7 @@ class projectlist extends base{ $(\'.' . $sTrClass . '\').removeClass("trproject-textfilter"); $(\'.' . $sTrClass . '\').removeClass("trproject-progressfilter"); $(\'.' . $sTrClass . '\').removeClass("trproject-projectfilter"); + $(\'.' . $sTrClass . '\').removeClass("trproject-projectgroupfilter"); $(\'.' . $sTrClass . '\').removeClass("trprojectfiltered"); $(\'button.prjprogress\').removeClass(\'selected\'); @@ -217,6 +235,10 @@ class projectlist extends base{ var sPrj=$("#prjfilter").val(); localStorage.setItem("selectedPrj", sPrj); + // --- get project group filter + var sPrjGroup=$("#prjgroupfilter").val(); + localStorage.setItem("selectedPrjGroup", sPrjGroup); + // --- get progress filter var sProgress=$("#progressfilter").val(); localStorage.setItem("progress", sProgress); @@ -236,6 +258,13 @@ class projectlist extends base{ $(this).addClass("trproject-projectfilter"); } } + if (sPrjGroup){ + if ($(this).hasClass("group-"+sPrjGroup)){ + $(this).addClass("trprojectgroupfiltered"); + } else { + $(this).addClass("trproject-projectgroupfilter"); + } + } if (sProgress && !$(this).hasClass("progress" + sProgress)){ $(this).addClass("trproject-progressfilter"); } @@ -314,6 +343,13 @@ class projectlist extends base{ $("#efilter").val(localStorage.getItem("efilter")); window.setTimeout("filterTableByTyping();", 10); } + + if(localStorage.getItem("selectedPrjGroup") && localStorage.getItem("selectedPrjGroup")!=\'null\'){ + $("#prjgroupfilter").val(localStorage.getItem("selectedPrjGroup")); + } else { + // $("#selectedPrjGroup").val($("#selectedPrjgroup option:first").val()); + } + // window.setTimeout("filterOverviewTable();", 10); filterOverviewTable(); @@ -348,6 +384,7 @@ class projectlist extends base{ function showResetbtn(){ var sVisible=$("#efilter").val()?"visible":"hidden"; if ($("#prjfilter").val()>"")sVisible="visible"; + if ($("#prjgroupfilter").val()>"")sVisible="visible"; // if ($("#phasefilter").val()!="' . $sColClass . '")sVisible="visible"; if ($("#rolefilter").val())sVisible="visible"; if ($("#progressfilter").val())sVisible="visible"; @@ -361,7 +398,8 @@ class projectlist extends base{ * Aktion des Filter reset: Filter zurücksetzen */ function resetFilter(){ - $(\'#prjfilter\').val(\'\'); + $("#prjfilter").val(""); + $("#prjgroupfilter").val(""); // $("#phasefilter").val(""); filterOverviewTable(); $("#efilter").val(""); @@ -421,6 +459,14 @@ class projectlist extends base{ <button id="btnProgressinprogress" class="btn btn-default prjprogress" onclick="$(\'#progressfilter\').val(\'inprogress\'); return setprogress();" >'.$sDivInprogress.'<span>'.$iInprogress.'</span></button> <button id="btnProgresshasqueue" class="btn btn-default prjprogress" onclick="$(\'#progressfilter\').val(\'hasqueue\'); return setprogress();" >'.$sDivHasqueue .'<span>'.$iHasqueue.'</span></button> + + <label for="prjgroupfilter"> + ' . t("projectgroup") . ': + </label> + <select id="prjgroupfilter" class="form-control" onchange="filterOverviewTable(); return false;"> + ' . $sPrjGroupFilter . ' + </select> + <span style="display: none;"> <span class="view viewextended"> @@ -431,6 +477,7 @@ class projectlist extends base{ ' . $sPrjFilter . ' </select> + <span style="display: none;"> <label for="phasefilter">Phasen:</label> diff --git a/public_html/deployment/classes/rollout_base.class.php b/public_html/deployment/classes/rollout_base.class.php index 2a7e6017503c4448a92fa0b39fff4335b929dc67..2f022d1683aec7ed5348eed311ed07c45dcde77e 100644 --- a/public_html/deployment/classes/rollout_base.class.php +++ b/public_html/deployment/classes/rollout_base.class.php @@ -3,8 +3,8 @@ require_once 'rollout.interface.php'; require_once 'cache.class.php'; /** - * rollout_base class that will beextended in a rollout plugin - * + * rollout_base class that will be extended in a rollout plugin + * see deployment/plugins/rollout/* * * @author axel */ @@ -30,13 +30,20 @@ class rollout_base implements iRolloutplugin{ * @var type */ protected $_aLang=false; - + /** * set language; 2 letter code, i.e. "de"; default language is "en" ; a * file "lang_en.json" is required in the plugin dir * @var string */ - protected $_sLang = 'en'; + protected $_sFallbackLang = 'en-en'; + + /** + * set language; 2 letter code, i.e. "de"; default language is "en" ; a + * file "lang_en.json" is required in the plugin dir + * @var string + */ + protected $_sLang = 'en-en'; /** * string with phase of project; one of preview|stage|live @@ -157,7 +164,7 @@ class rollout_base implements iRolloutplugin{ $sReturn=''; $sKeyPrefix=$this->getId().'_'.$sKey; - $oForm = new formgen($aForms); + $oForm = new formgen(); foreach ($aFormdata as $elementData) { $elementKey=$sKeyPrefix.'_'.$i++; $sReturn.=$oForm->renderHtmlElement($elementKey, $elementData); @@ -336,7 +343,7 @@ class rollout_base implements iRolloutplugin{ * translated texts can be done with $this->_t("your_key") * * @see _t() - * @param string $sLang language code, i.e. "de" + * @param string $sLang language code, i.e. "de-de" * @return boolean */ public function setLang($sLang=false){ @@ -344,6 +351,10 @@ class rollout_base implements iRolloutplugin{ $oReflection=new ReflectionClass($this); $sFile=dirname($oReflection->getFileName()) . '/lang_'.$this->_sLang.'.json'; + if (!file_exists($sFile)){ + $sFile=dirname($oReflection->getFileName()) . '/lang_'.$this->_sFallbackLang.'.json'; + $this->_sLang=$this->_sFallbackLang; + } $this->_aLang=(file_exists($sFile)) ? json_decode(file_get_contents($sFile), 1) : $this->_aLang; return true; } diff --git a/public_html/deployment/classes/sws.class.php b/public_html/deployment/classes/sws.class.php index 8fcbd7dacfb2eadde41b4674a065662ab2dba340..bc1c406775d2c24a9861f23b1da6db1ba89f061a 100644 --- a/public_html/deployment/classes/sws.class.php +++ b/public_html/deployment/classes/sws.class.php @@ -173,7 +173,7 @@ class sws { * @return boolean */ private function _verifyParamValue($sParamValue){ - $sOKChars='a-z0-9\"\{\}\[\]\.\,\ \:\-\+'; + $sOKChars='a-z0-9\"\{\}\[\]\.\,\ \:\-\+\_'; /* $sOKChars='a-z0-9\"\`\'\{\}\[\]\.\,\ \:\-\+' .'\<\>\=' diff --git a/public_html/deployment/classes/user.class.php b/public_html/deployment/classes/user.class.php index 285834040774a04fd38d294710f4128400e4c4b3..33ac0c66f2d1e4eaf776fa562fd6851f5efe0510 100644 --- a/public_html/deployment/classes/user.class.php +++ b/public_html/deployment/classes/user.class.php @@ -72,11 +72,19 @@ class user { // UNUSED SO FAR private function _getUser2Projects(){ - return require(__DIR__ . '/../../../config/inc_user2projects.php'); + $sFile=__DIR__ . '/../../../config/inc_user2projects.php'; + return file_exists($sFile) + ? require $sFile + : [] + ; } private function _getUser2Roles(){ - return require(__DIR__ . '/../../../config/inc_user2roles.php'); + $sFile=__DIR__ . '/../../../config/inc_user2roles.php'; + return file_exists($sFile) + ? require $sFile + : ['admin'=>['admin']] + ; } /** * TODO: reimplement @@ -138,11 +146,11 @@ class user { public function authenticate(){ global $aConfig, $aParams; //print_r($aConfig); - if(!array_key_exists('auth', $aConfig) || !count($aConfig['auth']) || !array_key_exists('user', $aParams)){ + if(!isset($aConfig['auth']) || !is_array($aConfig['auth']) || !count($aConfig['auth']) || !isset($aParams['user'])){ return false; } $sUser=$aParams['user']; - $sPassword=array_key_exists('password', $aParams)?$aParams['password']:false; + $sPassword=isset($aParams['password']) ? $aParams['password'] : false; foreach (array_keys($aConfig['auth']) as $sAuthMethod){ $oUserAuth=false; @@ -201,9 +209,8 @@ class user { */ public function showDenied(){ return '<div class="alert alert-danger" role="alert">' - . ($this->_sUsername ? ' User: '.$this->_sUsername : '') . ($this->_sUsername - ? t("class-user-error-deny-no-role").'<br>('.$this->_sLastCheckedPermission.')' + ? t("class-user-error-deny-no-role").'<br>'.$this->_sUsername.' --> ('.$this->_sLastCheckedPermission.')<br>' : t("class-user-error-login-required") ) . '</div><br>' diff --git a/public_html/deployment/classes/vcs.git.class.php b/public_html/deployment/classes/vcs.git.class.php index d8d50f9872181d3faea4f85c93711d97f332ca85..24c96d16e76f58568bc68e20ec582df32d76aff9 100644 --- a/public_html/deployment/classes/vcs.git.class.php +++ b/public_html/deployment/classes/vcs.git.class.php @@ -46,7 +46,7 @@ class vcs implements iVcs { /** * name of the default remote branch to access - * @var type + * @var string */ private $_sCurrentBranch = "origin/master"; @@ -80,7 +80,7 @@ class vcs implements iVcs { // checks // foreach (array("type", "url") as $key) { foreach (array("type") as $key) { - if (!array_key_exists($key, $aRepoConfig)) { + if (!isset($aRepoConfig[$key])) { die("ERROR: key $key does not exist in config <pre>" . print_r($aRepoConfig, true) . "</pre>"); } if (!$aRepoConfig[$key]) { @@ -120,7 +120,9 @@ class vcs implements iVcs { $sGitCmd.='git remote add origin "' . $this->getUrl() . '" 2>&1 '; // $sGitCmd='time ('.$sGitCmd.')'; // exec($sGitCmd, $aOutput, $iRc); - exec($sGitCmd); + $this->log(__FUNCTION__." start command <code>$sGitCmd</code>"); + exec($sGitCmd, $aOutput,$iRc); + $this->log(__FUNCTION__." command ended with rc=$iRc ". '<pre>'.implode("\n", $aOutput).'</pre>', ($iRc==0 ? 'info':'error')); } return $this->_sTempDir; } @@ -346,9 +348,9 @@ class vcs implements iVcs { if ( is_array($this->_aRemoteBranches) && ( - array_key_exists($sBranch, $this->_aRemoteBranches) && $sVerifyRevision && $this->_aRemoteBranches[$sBranch]['revision'] == $sVerifyRevision + isset($this->_aRemoteBranches[$sBranch]) && $sVerifyRevision && $this->_aRemoteBranches[$sBranch]['revision'] == $sVerifyRevision || - array_key_exists($sBranch, $this->_aRemoteBranches) && !$sVerifyRevision + isset($this->_aRemoteBranches[$sBranch]) && !$sVerifyRevision ) ) { // it is up to date - doing nothing @@ -408,6 +410,14 @@ class vcs implements iVcs { } $sGitCmd = 'export GIT_SSH="' . $this->_sWrapper . '" ; export PKEY="' . $this->_sKeyfile . '" ; '; + + // Luki: + // git clone -b <branch_or_tag> --single-branch <repo_url> --depth 1 --bare <dir> + /* + $sWorkDir=$sWorkDir ? $sWorkDir : $this->_sTempDir; + $sWorkDir='/dev/shm/abc'; + $sGitCmd.='git clone -b '.$this->_sCurrentBranch.' --single-branch '.$this->getUrl().' --depth 1 --bare "' . $sWorkDir . '" 2>&1; rm -rf "' . $sWorkDir . '"'; + */ if ($sWorkDir) { $sGitCmd.='cd "' . $sWorkDir . '" && '; } else { @@ -421,8 +431,9 @@ class vcs implements iVcs { // TODO: git 1.9 does needs only the line with --tags $sGitCmd.=' ( ' + // . 'git fetch --update-head-ok --tags --depth 1 2>&1 ; ' // 1.5 s . 'git fetch --update-head-ok --tags --depth 1 2>&1 ; ' // 1.5 s - . 'git fetch --update-head-ok --depth 1 2>&1 ' // 1.5 s + //. 'git fetch --update-head-ok --depth 1 2>&1 ' // 1.5 s . ') && '; } diff --git a/public_html/deployment/inc_functions.php b/public_html/deployment/inc_functions.php index ec157dd376d936267b7f033abe171e510f4e959a..aa27cdd8ace3d0e84199dd302145f87ab6181a5f 100644 --- a/public_html/deployment/inc_functions.php +++ b/public_html/deployment/inc_functions.php @@ -118,7 +118,10 @@ if (isset($_SERVER) && is_array($_SERVER) && array_key_exists("REQUEST_URI", $_S */ foreach (array_keys($aParams) as $sKey) { - $aParams[$sKey] = str_replace(array('\\', "\0", "\n", "\r", "'", '"', "\x1a"), array('\\\\', '\\0', '\\n', '\\r', "\\'", '\\"', '\\Z'), $aParams[$sKey]); + $aParams[$sKey] = is_string($aParams[$sKey]) + ? str_replace(array('\\', "\0", "\n", "\r", "'", '"', "\x1a"), array('\\\\', '\\0', '\\n', '\\r', "\\'", '\\"', '\\Z'), $aParams[$sKey]) + : $aParams[$sKey] + ; } } @@ -203,7 +206,7 @@ function getTopArea() { $sMyPhase = "[phase]"; $sMyRev = " [no rev] "; // bug#1511 json file was moved 1 level up - $sJsonfile = __DIR__ . "/../../ci-webgui.json"; + $sJsonfile = dirname(dirname(__DIR__)) . "/ciserver.json"; if (file_exists($sJsonfile)) { $aJson = json_decode(file_get_contents($sJsonfile), true); if (array_key_exists("date", $aJson)) diff --git a/public_html/deployment/index.php b/public_html/deployment/index.php index 8dc4e61393400a55bdf3a9678f58aa7ae209b9e4..c9a84309e2702e4442791710afb4a84c1aba8be0 100644 --- a/public_html/deployment/index.php +++ b/public_html/deployment/index.php @@ -23,6 +23,10 @@ ini_set('display_startup_errors', 1); error_reporting(E_ALL); require_once("./classes/page.class.php"); + +// detect first run +$bFirstRun=!file_exists("../../config/config_custom.php") || !file_exists("../../config/inc_user2roles.php"); + require_once("../../config/inc_projects_config.php"); require_once("./classes/logger.class.php"); require_once("./classes/user.class.php"); @@ -52,7 +56,10 @@ $oCLog->add("parsing params " . '<pre>aParams: '.print_r($aParams, true).'</pre>' ); - +if($bFirstRun){ + $sAction='installer'; +} + // ------ Ausgabe $sHeader = '<style>'; foreach ($aConfig["phases"] as $sPhase => $aData) { @@ -62,10 +69,16 @@ foreach ($aConfig["phases"] as $sPhase => $aData) { } $sHeader.='</style>'; $sTopArea=getTopArea(); +$sBanner=isset($aConfig['banner']) && $aConfig['banner'] ? '<div class="alert alert-info">'.$aConfig['banner'].'</div>' : ''; $sTopAction=getAction(); // ------ action $oUser=new user(); +if (isset($aConfig["auth"]['forceuser']) && $aConfig["auth"]['forceuser']){ + $oCLog->add("Found config -> auth -> forceuser: using fake identity [".$aConfig["auth"]['forceuser'].']', "warning"); + $oUser->setUser($aConfig["auth"]['forceuser']); +} + if ($oUser->hasPermission('page_'.$sAction)){ $sActionFile = __DIR__ . '/pages/act_' . $sAction . ".php"; @@ -99,18 +112,20 @@ if ($oUser->hasPermission('page_'.$sAction)){ $oCLog->add("Finally: rendering page ..."); $sPhpOut = ' - <div id="header" style="display: none;"> - IML DEPLOYMENT GUI - </div> <br> - ' . $sTopArea . ' + ' . $sTopArea .' <div id="content"> - ' . $sTopAction . ' + ' . $sBanner . $sTopAction . ' ' . $sPhpOut . ' </div> <div id="footer"> - IML Deployment © 2015-' . date("Y") . ' <a href="https://gitlab.iml.unibe.ch/admins/imldeployment/tree/master" target="_blank">Institut für medizinische Lehre; Universität Bern</a> + '.t("menu-brand").' © 2013-' . date("Y") . ' <a href="https://git-repo.iml.unibe.ch/iml-open-source/imldeployment/" target="_blank">Institut für Medizinische Lehre; Universität Bern</a> </div> + + <!-- + <script src="/deployment/plugins/shellcmd/load/render.js" /> + --> + '.$oCLog->render(); $oPage = new Page(); diff --git a/public_html/deployment/main.css b/public_html/deployment/main.css index 0ccffde211603aaa43534bdef952a36af4edaf0f..ced1a280cdfe1d26d36e6f61f1b178ab311f1d44 100644 --- a/public_html/deployment/main.css +++ b/public_html/deployment/main.css @@ -1,9 +1,10 @@ -body{padding-top: 0;} +body{padding-top: 0; + background: linear-gradient(-20deg, #fff 10%,#dde,#fff 90%) fixed; +} #header,#footer{ - background:#eee; background: linear-gradient(#fff,#eee); + background:#eee; padding: 1em; font-size: 300%; color:#a33; - text-shadow: 1px 1px 0 #fff, 10px 10px 10px #aaa; border-bottom: 2px solid #ccc; } #footer{ @@ -11,12 +12,14 @@ body{padding-top: 0;} font-size: 100%; color:#a33; text-align: right; - text-shadow: 1px 1px 0 #fff, 3px 3px 3px #aaa; border: 0px solid #ccc; border-top: 0px solid #ddd; margin-top: 5em; } -.navbar {background:#023; border-radius: 0; position: fixed; top: 0px; width: 100%; left: 0; z-index: 100;} +.navbar {background:#436; border-radius: 0; position: fixed; top: 0px; width: 100%; left: 0; z-index: 100;} +.navbar-inverse .navbar-nav > li > a { + color: #fff; +} .navbar-inner {padding: 0 6em;box-shadow: 0 0 3em #888;} #header2{ background: none; @@ -25,26 +28,31 @@ body{padding-top: 0;} } -.description{font-weight:bold; color:#ccc; font-size: 150%; font-style: italic;} -.navbar-brand {color:#a33 !important;} +.description{font-weight:bold; color:rgba(0,0,0,0.3); font-size: 150%; font-style: italic;} +.navbar-brand {font-size: 150%;} .navbar-inverse .navbar-brand {color:#ddd !important;} -.imllogo:before{background: rgb(255,0,51); color:#fff; padding: 0.5em 0.3em; content: 'IML'; font-weight: bold; } +.imllogo:before{background: rgb(255,0,51); color:#fff; padding: 0.5em 0.4em; content: 'IML'; font-weight: bold; } + +.navbar-nav > li > .dropdown-menu { + overflow: scroll; + max-height: 50em; +} #swversion {float: right; right: 0em; position: fixed; top: 4em; padding: 0 0.5em; - background: #eee; color:#888; + background: rgba(0,0,0,0.05); color:#555; border-bottom-left-radius: 0.5em; } #content{ + background: #fff; margin: 0 2% 0; - border-left: 0px solid #ccc; - padding: 1em; - box-shadow: 0 0 15px #eee; - padding: 0.5em; - box-shadow: 0 0 1em #ddd; + border: 1px solid rgba(0,0,0,0.06); + /* + box-shadow: 1em 1em 2em #fff; + */ } div#navtop, div#navbuttom{background: #dde; padding: 0.5em;} @@ -131,7 +139,7 @@ tr:hover{background:#ddd; background: linear-gradient(#f4f4f4,#fff,#f4f4f4);} .trprojectfiltered a.btn, .trproject:hover a.btn{opacity: 1;} -.trproject{border-left:0.5em solid #eee;} +.trproject{border-left:0.5em solid rgba(0,0,0,0);} tr.progressinprogress{border-left:0.5em solid #8db;} div.progressinprogress{color: #6a9;} tr.progresshasqueue{border-left:0.5em solid #f81;} @@ -149,6 +157,7 @@ button.prjprogress.selected{background:#f4f4f4; box-shadow: 0 0 1em #ddd inset; .trproject-textfilter, .trproject-projectfilter, +.trproject-projectgroupfilter, .trproject-progressfilter { display: none; diff --git a/public_html/deployment/pages/act_build.php b/public_html/deployment/pages/act_build.php index ca11ba0c60f475e779a37d433915db09591b5a10..2dab289d0406701b30a372b7864f36f6c6f202e3 100644 --- a/public_html/deployment/pages/act_build.php +++ b/public_html/deployment/pages/act_build.php @@ -14,6 +14,7 @@ require_once("./classes/project.class.php"); require_once("./classes/formgen.class.php"); +set_time_limit(0); // --- Checks $oPrj = new project($aParams["prj"]); @@ -158,7 +159,7 @@ if (!array_key_exists("confirm", $aParams)) { $sOut.= '<div id="' . $sDivname . '"></div>' . '<script> - var iRepeat=3000; + var iRepeat=2000; // start build process $.post( "' . $sUrlStartAction . '", function( data ) { diff --git a/public_html/deployment/pages/act_installer.php b/public_html/deployment/pages/act_installer.php new file mode 100644 index 0000000000000000000000000000000000000000..bb871eeace828623001520cad076457533a846ef --- /dev/null +++ b/public_html/deployment/pages/act_installer.php @@ -0,0 +1,37 @@ +<?php + +/* ###################################################################### + + IML DEPLOYMENT + + INSTALLER + + --------------------------------------------------------------------- + 2022-07-25 Axel <axel.hahn@iml.unibe.ch> + ###################################################################### */ + +$sOut = '' +.' + +<h2>Welcome to the IML CI SERVER!</h2> + +<p> + This page appears on the first run. Or better: as long no configuration exists.<br> + + <br> + <br> + Go to the directory <code>'.dirname($_SERVER['DOCUMENT_ROOT']).'/config/</code>.<br> + <br> + Copy 2 <code>*.dist</code> files to <code>config_custom.php</code> and <code>inc_user2roles.php</code>.<br> + <br> + <a href="?" class="btn btn-default">Reload</a><br> + <br> +</p> +<script> + window.setTimeout("location.reload()", 5000); +</script> +' +; + +// -- Ausgabe +echo $sOut; diff --git a/public_html/deployment/pages/act_overview.php b/public_html/deployment/pages/act_overview.php index 9336f049f66d1ead20319406b563738beff6337a..0cfe8e5c297800d51b10808768cef49c021fc6bd 100644 --- a/public_html/deployment/pages/act_overview.php +++ b/public_html/deployment/pages/act_overview.php @@ -59,7 +59,7 @@ if (!array_key_exists("prj", $aParams)) { } else { $sBuildErrorContent=$oHtml->getBox('info', t('build-failes-none')); } - + $sListOfBranches='<h4>'.t('branch-select').'</h4><ol>'; foreach($oPrj->getRemoteBranches() as $aBranch){ $sListOfBranches.='<li title="'.$aBranch['revision'].'">'.$aBranch['label'] . '</li>'; @@ -89,12 +89,10 @@ if (!array_key_exists("prj", $aParams)) { </div> <div class="tab-pane" id="tab2"> <h3 id="h3repo">' . $oHtml->getIcon('repository') . t("repositoryinfos") . '</h3> - <div style="max-width: 40em;">' - . $oPrj->renderRepoInfo() - // . $oPrj->renderSelectRemoteBranches() - . $sListOfBranches . ' - ' . ($oPrj->canAcceptPhase() ? '<br>'.$oPrj->renderLink("build") : '') . ' - </div> + ' . ($oPrj->canAcceptPhase() ? '<br>'.$oPrj->renderLink("build") : '') + // . $oPrj->renderRepoInfo() + // . $oPrj->renderSelectRemoteBranches() + . $sListOfBranches . ' </div> <div class="tab-pane" id="tab3"> <h3 id="h3versions">' . $oHtml->getIcon('sign-error') . t("build-failes") . '</h3> diff --git a/public_html/deployment/pages/act_setup.php b/public_html/deployment/pages/act_setup.php index 8a762aa199d9d5b1334bf7992c96fcddb786543c..8d9f8f4c20eb7211f7dc8199de394be4f71b0f0a 100644 --- a/public_html/deployment/pages/act_setup.php +++ b/public_html/deployment/pages/act_setup.php @@ -111,12 +111,6 @@ if ($aParams["prj"] == "all") { '["builtsToKeep"]'=>array('type'=>'text', 'validate'=>'isinteger'), '["lang"]'=>array('type'=>'text'), ), - 'install'=>array( - '["installPackages"]["user"]'=>array('type'=>'text'), - '["installPackages"]["addkeycommand"]'=>array('type'=>'text'), - '["installPackages"]["testcommand"]'=>array('type'=>'text'), - '["installPackages"]["command"]'=>array('type'=>'text'), - ), ); foreach ($aConfig['phases'] as $sPhase => $aPhaseData){ $aMapping['phase-'.$sPhase]=array( diff --git a/public_html/deployment/plugins/build/tgz/build_tgz.php b/public_html/deployment/plugins/build/tgz/build_tgz.php new file mode 100644 index 0000000000000000000000000000000000000000..743a570b348326cc4ffe2a5766e70769c7d1e686 --- /dev/null +++ b/public_html/deployment/plugins/build/tgz/build_tgz.php @@ -0,0 +1,32 @@ +<?php + +/** + * + * Build plugin - TGZ + * + * @author <axel.hahn@iml.unibe.ch> + */ +class build_tgz extends build_base { + + /** + * check requirements if the plugin could work + * @return array + */ + public function checkRequirements() { + return [ + 'which tar' + ]; + } + + /** + * get an array with shell commands to execute + * @return array + */ + public function getBuildCommands(){ + return [ + 'cd "'. $this->getBuildDir(). '" && tar -czf "'. $this->getOutfile().'" .' + ]; + } + + +} diff --git a/public_html/deployment/plugins/build/tgz/info.json b/public_html/deployment/plugins/build/tgz/info.json new file mode 100644 index 0000000000000000000000000000000000000000..dfff108b663107a81a2d0dfb206b444fbcb365af --- /dev/null +++ b/public_html/deployment/plugins/build/tgz/info.json @@ -0,0 +1,11 @@ +{ + "name": "TGZ Archive", + "description": "Create an TGZ archive", + "author": "Axel Hahn; University of Bern; Institute for Medical education", + + "version": "1.0", + "url": "[included]", + "license": "GNU GPL 3.0", + + "extension": "tgz" +} diff --git a/public_html/deployment/plugins/build/tgz/lang_de-de.json b/public_html/deployment/plugins/build/tgz/lang_de-de.json new file mode 100644 index 0000000000000000000000000000000000000000..33c3636fb2b9d9e6682b45cb15cb529accdd16b5 --- /dev/null +++ b/public_html/deployment/plugins/build/tgz/lang_de-de.json @@ -0,0 +1,4 @@ +{ + "plugin_name": "TGZ", + "description": "TGZ Archiv erstellen" +} \ No newline at end of file diff --git a/public_html/deployment/plugins/build/tgz/lang_en-en.json b/public_html/deployment/plugins/build/tgz/lang_en-en.json new file mode 100644 index 0000000000000000000000000000000000000000..efd39e8797f57b66880961193a6afc36881ac7c5 --- /dev/null +++ b/public_html/deployment/plugins/build/tgz/lang_en-en.json @@ -0,0 +1,4 @@ +{ + "plugin_name": "TGZ", + "description": "Create an TGZ archive" +} \ No newline at end of file diff --git a/public_html/deployment/plugins/build/zip/build_zip.php b/public_html/deployment/plugins/build/zip/build_zip.php new file mode 100644 index 0000000000000000000000000000000000000000..2fadc359a115da1f020de2bb93ee7f625db53226 --- /dev/null +++ b/public_html/deployment/plugins/build/zip/build_zip.php @@ -0,0 +1,36 @@ +<?php + +/** + * + * Build plugin - TGZ + * + * @author <axel.hahn@iml.unibe.ch> + */ +class build_zip extends build_base { + + /** + * check requirements if the plugin could work + * @return array + */ + public function checkRequirements() { + return [ + 'which zip' + ]; + } + + /** + * get an array with shell commands to execute + * used zip params: + * -9 compress better + * -q quiet operation + * -r recurse into directories + * @return array + */ + public function getBuildCommands(){ + return [ + 'cd "'. $this->getBuildDir(). '" && zip -9qr "'. $this->getOutfile().'" .' + ]; + } + + +} diff --git a/public_html/deployment/plugins/build/zip/info.json b/public_html/deployment/plugins/build/zip/info.json new file mode 100644 index 0000000000000000000000000000000000000000..0d4f6b9a8c0f016628833bade27240962a1dc3cc --- /dev/null +++ b/public_html/deployment/plugins/build/zip/info.json @@ -0,0 +1,11 @@ +{ + "name": "ZIP Archive", + "description": "Create a ZIP archive", + "author": "Axel Hahn; University of Bern; Institute for Medical education", + + "version": "1.0", + "url": "[included]", + "license": "GNU GPL 3.0", + + "extension": "zip" +} diff --git a/public_html/deployment/plugins/build/zip/lang_de-de.json b/public_html/deployment/plugins/build/zip/lang_de-de.json new file mode 100644 index 0000000000000000000000000000000000000000..f0bf38953e92764fb46594a7d33088ef508a17c5 --- /dev/null +++ b/public_html/deployment/plugins/build/zip/lang_de-de.json @@ -0,0 +1,4 @@ +{ + "plugin_name": "ZIP", + "description": "ZIP Archiv erstellen" +} \ No newline at end of file diff --git a/public_html/deployment/plugins/build/zip/lang_en-en.json b/public_html/deployment/plugins/build/zip/lang_en-en.json new file mode 100644 index 0000000000000000000000000000000000000000..c0fd28c102ae6d2ff87694ab729740c5f6aa339e --- /dev/null +++ b/public_html/deployment/plugins/build/zip/lang_en-en.json @@ -0,0 +1,4 @@ +{ + "plugin_name": "ZIP", + "description": "Create a ZIP archive" +} \ No newline at end of file diff --git a/public_html/deployment/plugins/rollout/awx/lang_de.json b/public_html/deployment/plugins/rollout/awx/lang_de-de.json similarity index 100% rename from public_html/deployment/plugins/rollout/awx/lang_de.json rename to public_html/deployment/plugins/rollout/awx/lang_de-de.json diff --git a/public_html/deployment/plugins/rollout/awx/lang_en.json b/public_html/deployment/plugins/rollout/awx/lang_en-en.json similarity index 100% rename from public_html/deployment/plugins/rollout/awx/lang_en.json rename to public_html/deployment/plugins/rollout/awx/lang_en-en.json diff --git a/public_html/deployment/plugins/rollout/awx/rollout_awx.php b/public_html/deployment/plugins/rollout/awx/rollout_awx.php index c9b315e1d7c7178ad2bdd6f35fba38db75581a82..41919f55bd53d82f3d09614f4add82c720eb5577 100644 --- a/public_html/deployment/plugins/rollout/awx/rollout_awx.php +++ b/public_html/deployment/plugins/rollout/awx/rollout_awx.php @@ -87,7 +87,7 @@ class rollout_awx extends rollout_base { * [id] => array('value' => [ID], 'label' => [NAME] [ID]) * @return array */ - static public function getAwxInventories(){ + public function getAwxInventories(){ $aResponse=$this->_httpRequest(array( 'url'=>'/inventories/?order_by=name'.$this->_sAwxApiPaging, 'method'=>'GET', @@ -128,7 +128,7 @@ class rollout_awx extends rollout_base { * [id] => array('value' => [ID], 'label' => [PLAYBOOK] [ID]) * @return array */ - static public function getAwxJobTemplates(){ + public function getAwxJobTemplates(){ $aResponse=$this->_httpRequest(array( 'url'=>'/job_templates/?order_by=name'.$this->_sAwxApiPaging, 'method'=>'GET', @@ -177,8 +177,8 @@ class rollout_awx extends rollout_base { // ----- Checks: $sCmdChecks=''; if($aConfig['extravars']){ - $aTmp=json_decode($aConfig['extravars']); - if (!$aTmp || !count($aTmp) ){ + $aTmp=json_decode($aConfig['extravars'], 1); + if (!$aTmp || !is_array($aTmp) || !count($aTmp) ){ $sCmdChecks.='echo "ERROR: Value in extravars has wrong Syntax - this is no JSON: '.$aConfig['extravars'].'"; exit 1; '; } $aConfig['extravars']=json_encode($aTmp); diff --git a/public_html/deployment/plugins/rollout/default/lang_de.json b/public_html/deployment/plugins/rollout/default/lang_de-de.json similarity index 100% rename from public_html/deployment/plugins/rollout/default/lang_de.json rename to public_html/deployment/plugins/rollout/default/lang_de-de.json diff --git a/public_html/deployment/plugins/rollout/default/lang_en.json b/public_html/deployment/plugins/rollout/default/lang_en-en.json similarity index 100% rename from public_html/deployment/plugins/rollout/default/lang_en.json rename to public_html/deployment/plugins/rollout/default/lang_en-en.json diff --git a/public_html/deployment/plugins/rollout/default/rollout_default.php b/public_html/deployment/plugins/rollout/default/rollout_default.php index a98ca37ace98359cca07afc6bb9d46020a8ac266..cc4941761eec4b2774da689b856bbba425ca71a2 100644 --- a/public_html/deployment/plugins/rollout/default/rollout_default.php +++ b/public_html/deployment/plugins/rollout/default/rollout_default.php @@ -25,6 +25,18 @@ class rollout_default extends rollout_base { return true; } + /** + * get array with commands to execute to deploy a package + * + * @param string $sPhase phase + * @return array + */ + public function getDeployCommands($sPhase){ + return [ + 'echo "SKIP"' + ]; + } + /** * override general form renderer: show a single message that no * configuration items exist diff --git a/public_html/deployment/plugins/rollout/ssh/lang_de.json b/public_html/deployment/plugins/rollout/ssh/lang_de-de.json similarity index 100% rename from public_html/deployment/plugins/rollout/ssh/lang_de.json rename to public_html/deployment/plugins/rollout/ssh/lang_de-de.json diff --git a/public_html/deployment/plugins/rollout/ssh/lang_en.json b/public_html/deployment/plugins/rollout/ssh/lang_en-en.json similarity index 100% rename from public_html/deployment/plugins/rollout/ssh/lang_en.json rename to public_html/deployment/plugins/rollout/ssh/lang_en-en.json diff --git a/public_html/deployment/plugins/shellcmd/getdata.php b/public_html/deployment/plugins/shellcmd/getdata.php new file mode 100644 index 0000000000000000000000000000000000000000..12b260e86700c3dbb0e1cd3a20868ea1bd3fdd4c --- /dev/null +++ b/public_html/deployment/plugins/shellcmd/getdata.php @@ -0,0 +1,139 @@ +<?php +/* + * script to be used as ajax poll request to get current status of an action + */ + +header('Content-Type: application/json'); +require_once('plugins_shellcmd.class.php'); + +$oShell=new shellcmd(); +$oShell->sendResponse(); + +/* + +// width of load=1 +$iMaxWidth=100; + +$sPlugin=isset($_GET['plugin']) && $_GET['plugin'] ? preg_replace('/^a-z0-9/', '', $_GET['plugin']) : false; + +// ---------------------------------------------------------------------- +// ---------------------------------------------------------------------- +function execCommand($sCmd){ + echo "DEBUG: ".__FUNCTION__ . "($sCmd)<br>"; + exec($sCmd, $aOut, $iResult); + return [ + 'command'=>$sCmd, + 'exitcode'=>$iResult, + 'output'=>$aOut, + ]; +} + + +if (!$sPlugin){ + echo "DEBUG: Missing param for a plugin.<br>"; + return [ 'error' => 'Missing param for a plugin.' ]; +} + +$sPluginfile=$sPlugin.'/command.php'; +$sPluginclass='shellplugin_'.$sPlugin; + +echo "DEBUG: sPluginfile=$sPluginfile<br>"; + +if (!file_exists($sPluginfile)){ + echo "DEBUG: Plugin seems to be corrupt. File not found: '. $sPluginfile.'<br>"; + return [ 'error' => 'Plugin seems to be corrupt. File not found: '. $sPluginfile ]; +} + + +include($sPluginfile); + +$oPlugin=new $sPluginclass(); + +echo "DEBUG: sCmd=$sCmd<br>"; + +$aResult=execCommand($sCmd); +if (function_exists("parsedata")){ + $aResult=parsedata($aResult); +} + +echo '<pre>$aResult = '.print_r($aResult, 1).'<pre>'; +return $aResult; + + +// ---------------------------------------------------------------------- +function getLoad(){ + $sCmd='uptime'; + $aResult=execCommand($sCmd); + $aTmp1=explode(',', $aResult['output'][0]); + $aResult['data']=[ + 'uptime'=>$aTmp1[0], + 'users'=>$aTmp1[1], + 'load'=>preg_replace('/^.*:/', '', $aTmp1[2]), + 'load5'=>$aTmp1[3], + 'load15'=>$aTmp1[4], + ]; + return $aResult; +} + +function getProcesses($sFilter){ + $sRegex=$sFilter ? $sFilter : 'SomethingThatWontMatchInProcessList'; + $sCmd="ps -f --forest | egrep -v '($sRegex)' | fgrep -v 'ps -f' | fgrep -v grep"; + return execCommand($sCmd); +} + + +// ---------------------------------------------------------------------- +// ---------------------------------------------------------------------- +function load(){ + global $iMaxWidth; + $sReturn=''; + + $aData=getLoad()['data']; + + $iMaxLoad=round(max($aData['load'], $aData['load5'], $aData['load15']))+1; + $iScale=$iMaxWidth/$iMaxLoad; + + $sScalaPart=str_repeat('_', (round($iScale)-1)).'|'; + $sScala=str_repeat($sScalaPart, $iMaxLoad).$iMaxLoad; + + + $sBar1=str_repeat('#', (int)($aData['load']*$iScale)); + $sBar5=str_repeat(' ', (int)($aData['load5']*$iScale - 1)).'|'; + $sBar15=str_repeat(' ', (int)($aData['load15']*$iScale - 1)).'^'; + + $sReturn.= 'LOAD '.$aData['load'].' .. '.$aData['load5'].' .. '.$aData['load15'].'<br>' + . $sScala.'<br>' + . substr($sBar1, 0, $iMaxWidth).'<br>' + . substr($sBar5, 0, $iMaxWidth).'<br>' + . substr($sBar15, 0, $iMaxWidth).'<br>' + ; + + return $sReturn; +} + + +function processes(){ + + return 'PROCESSES:<br>' + .shell_exec("ps -f --forest | egrep -v '(apache|httpd)' | fgrep -v 'ps -f' | fgrep -v grep") + .'<hr>all processes:<br>' + .shell_exec("ps -f --forest") + ; +} + +// ---------------------------------------------------------------------- +// MAIN +// ---------------------------------------------------------------------- + +$sRemove='apache|httpd|php-fpm'; + +echo '<pre>' + . load() + // .'<hr size="1">' + + .'<hr size="1">' + .'API DATA' + .'<pre>getLoad()='.print_r(getLoad(), 1).'</pre>' + .'<pre>getProcesses(\''.$sRemove.'\')='.print_r(getProcesses($sRemove), 1).'</pre>' + ; +*/ \ No newline at end of file diff --git a/public_html/deployment/plugins/shellcmd/load/plugin.php b/public_html/deployment/plugins/shellcmd/load/plugin.php new file mode 100644 index 0000000000000000000000000000000000000000..ec9338047ae1d60dbb39120bad91a5ad9da444b6 --- /dev/null +++ b/public_html/deployment/plugins/shellcmd/load/plugin.php @@ -0,0 +1,70 @@ +<?php +/** + * + * SHELLCMD PLUGIN :: LOAD + * + * ---------------------------------------------------------------------- + * 2022-08-05 axel.hahn@iml.unibe.ch + */ +class shellcmd_load { + /** + * @var command line to exectute + */ + protected $_command='uptime'; + + + /** + * constructor ... returns command + * @return string + */ + public function __constructor(){ + return $this->getCommand(); + } + + /** + * get command + * @return string + */ + public function getCommand(){ + return $this->_command; + } + + + /** + * parse output and extract wanted values in section "data" + * @return array + */ + public function parsedata($aResult){ + $aTmp1=array_reverse(explode(',', $aResult['output'][0])); + // print_r($aTmp1); + $aResult['data']=[ + 'uptime'=>trim($aTmp1[4]), + 'users'=>trim(str_replace(' users', '', $aTmp1[3])), + 'load'=>trim(preg_replace('/^.*:/', '', $aTmp1[2])), + 'load5'=>trim($aTmp1[1]), + 'load15'=>trim($aTmp1[0]), + ]; + return $aResult; + } + +} +/* + +EXAMPLE OUTPUT + +{ + "command": "uptime", + "exitcode": 0, + "output": [ + " 13:36:54 up 1 day, 7:39, 0 users, load average: 1.18, 1.29, 1.33" + ], + "data": { + "uptime": "7:39", + "users": "0", + "load": "1.18", + "load5": "1.29", + "load15": "1.33" + } +} + +*/ \ No newline at end of file diff --git a/public_html/deployment/plugins/shellcmd/load/render.js b/public_html/deployment/plugins/shellcmd/load/render.js new file mode 100644 index 0000000000000000000000000000000000000000..f4d310608589579e60c7cdb0171f53e5b7ba5d79 --- /dev/null +++ b/public_html/deployment/plugins/shellcmd/load/render.js @@ -0,0 +1,7 @@ +function load_render(aData){ + var sReturn=''; + sReturn+=aData['command']+"<br>" + + "Load: "+aData["data"]['load']+"<br>" + ; + return sReturn; +} \ No newline at end of file diff --git a/public_html/deployment/plugins/shellcmd/plugins_shellcmd.class.php b/public_html/deployment/plugins/shellcmd/plugins_shellcmd.class.php new file mode 100644 index 0000000000000000000000000000000000000000..938c8c57c96560ba70d7b45f6f418571983140a5 --- /dev/null +++ b/public_html/deployment/plugins/shellcmd/plugins_shellcmd.class.php @@ -0,0 +1,103 @@ +<?php + +class shellcmd { + + protected $_sPlugin=false; + protected $_oPlugin=false; + protected $_aReturn=false; + + protected $_debug=false; + + /** + * constructor + * @return bool + */ + public function __construct(){ + return $this->detectPlugin(); + } + + + /** + * write debug output ... if enabled + */ + protected function _wd($s){ + echo $this->_debug ? 'DEBUG '.__CLASS__ . ' '.$s."<br>\n" : ''; + } + + /** + * detect plugin name to load from GET param "plugin" + */ + public function detectPlugin(){ + $this->_sPlugin=isset($_GET['plugin']) && $_GET['plugin'] ? preg_replace('/^a-z0-9/', '', $_GET['plugin']) : false; + $this->_wd("detected plugin: ".$this->_sPlugin); + return true; + } + + + /** + * initialize the shellcmd plugin + * @returm boolean + */ + protected function _loadPlugin(){ + $this->_oPlugin=false; + if (!$this->_sPlugin){ + $this->_wd("Missing param for a plugin"); + $this->_aReturn=[ 'error' => 'Missing param for a plugin.' ]; + return false; + } + $sPluginfile=$this->_sPlugin.'/plugin.php'; + $sPluginclass='shellcmd_'.$this->_sPlugin; + + if (!file_exists($sPluginfile)){ + $this->_wd("Plugin seems to be corrupt. File not found: '. $sPluginfile."); + $this->_aReturn=[ 'error' => 'Plugin seems to be corrupt. File not found: '. $sPluginfile ]; + return false; + } + include($sPluginfile); + $this->_oPlugin=new $sPluginclass(); + return true; + } + /** + * helper execute a given command and return array with executed + * command, returncode, output + * @return + */ + protected function _execCommand($sCmd){ + exec($sCmd, $aOut, $iResult); + return [ + 'command'=>$sCmd, + 'exitcode'=>$iResult, + 'output'=>$aOut, + ]; + } + + /** + * get data from plugin command and return array with executed + * command, returncode, output, parsed data + * @return array + */ + public function get(){ + $this->_loadPlugin(); + if (!$this->_oPlugin){ + return $this->_aReturn; + } + + $sCmd=$this->_oPlugin->getCommand(); + $this->_wd("sCmd=$sCmd"); + + $this->_aResult=$this->_execCommand($sCmd); + if (method_exists($this->_oPlugin, "parsedata")){ + $this->_aResult=$this->_oPlugin->parsedata($this->_aResult); + } + return $this->_aResult; + } + + /** + * send response as json + */ + public function sendResponse(){ + header('Content-Type: application/json'); + echo json_encode($this->get(), JSON_PRETTY_PRINT); + } + +} diff --git a/public_html/deployment/plugins/shellcmd/processes/command.php b/public_html/deployment/plugins/shellcmd/processes/command.php new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/readme.md b/readme.md new file mode 100644 index 0000000000000000000000000000000000000000..2d54e8b049259b7079e707d22565ed94623178c1 --- /dev/null +++ b/readme.md @@ -0,0 +1,40 @@ +# CI Server # + +Free software and Open Source from University of Bern :: IML - Institute of Medical Education + +📄 Source: <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment> \ +📜 License: GNU GPL 3.0 \ +📖 Docs: TODO + +- - - + +## Description ## + +CI node that checks out projects from git repositories and builds an deployable archive. +The archives can be synched to multiple deployment targets e.g. puppet master or a protected software archive. + +## Related projects ## + +* CI package server <https://git-repo.iml.unibe.ch/iml-open-source/ci-pkg> +* Deployment client written in bash <https://git-repo.iml.unibe.ch/iml-open-source/imldeployment-client> + +## Features ## + +* API to start a build from somewhere, e.g. from a devops workplace or Gitlab server +* checkout from git via SSH with multiple ssh keys (can be extended with a plugin) +* build has hooks to customize build process +* In our institute it builds projects written in + * PHP + * NodeJS - using NVM for custom Node versions + * Ruby - using RVM for custom Ruby versions +* sync built archives to deploy systems +* trigger rollout via ssh command or AWX API call (can be extended with a plugin) +* receives install status +* sends messages (email, Slack) + + +## Screenshot ## + +The overview over all projects is the starting page after login. It shows all projects and which build is rolled out to which phase. + + diff --git a/shellscripts/gitsshwrapper.sh b/shellscripts/gitsshwrapper.sh index 0643f535ae57200d880a9ee2d214de180c4d346e..01d04a45a240ab04328bbfdffd20bc480970f9fd 100644 --- a/shellscripts/gitsshwrapper.sh +++ b/shellscripts/gitsshwrapper.sh @@ -6,9 +6,12 @@ # gitwrapperssh.sh [keyfile] [command] # +# sshparams= +sshparams="-o StrictHostKeyChecking=no" + if [ -z "$PKEY" ]; then # if PKEY is not specified, run ssh using default keyfile - ssh "$@" + ssh "${sshparams}" "$@" else - ssh -i ${PKEY} "$@" + ssh "${sshparams}" -i ${PKEY} "$@" fi \ No newline at end of file