## CHECK_DOCKER_STATS Script: `check_docker_stats` **check_docker_stats** shows the docker resources of containers. This check sends performance data. ## Requirements * `docker` Docker must be installed * `jq` must be installed - commandline JSON processor * `bc` must be installed - an arbitrary precision calculator language * sudo permissions on docker command ```txt icingaclient ALL=(ALL) NOPASSWD:SETENV: /usr/bin/docker ``` ## Standalone installation From this repository ypu need next to this script: * `inc_pluginfunctions` shared function for all IML checks written in bash * `inc_dockerfunctions.sh` ## Rootless docker It works with docker setups as root and can handle rootless docker instances. To bring it up and running with an unpriviledged icinga user it must be able to access docker. Copy the docker binary eg. from /home/dockeruser/bin/docker to /usr/bin/. Then use the parameter `-p /usr/bin`. If DOCKER_HOST is not set then the docker socket is detected from a running docker instance and will be fetched from the process list. ## Syntax ```txt ______________________________________________________________________ CHECK_DOCKER_STATS v1.1 (c) Institute for Medical Education - University of Bern Licence: GNU GPL 3 https://os-docs.iml.unibe.ch/icinga-checks/Checks/check_docker_stats.html ______________________________________________________________________ Show resources of running docker instances. This check provides performance data except on mode "all". USAGE: check_docker_stats [OPTIONS] OPTIONS: General: -h, --help this help -d, --debug Debug; Flag: show docker env values. Connect to docker: -p, --path Custom directory for docker binary -t, --target Custom docker target; value for DOCKER_HOST Needed only if Docker does not run on a unix socket or multiple users run a rootless docker daemon. mode: -m, --mode MODE Mode what kind of information to show. The output of container list is sorted by the maximum value first. all default; This view is unsorted. This check has no performance data. cpu CPU usage total of all containers mem Memory max usage of most consuming container memtotal sum of memory usage netio sum of netio of all containers IN and OUT blockio sum of blockio of all containers IN and OUT EXAMPLES: check_docker_stats Show stats of all containers check_docker_stats -p /usr/bin Show stats of all containers. The docker binary will be searched in given path first - then in all other dirs of $PATH check_docker_stats -d Show stats of all containers and json with docker env infos. check_docker_stats -m cpu Show cpu usage total of all containers. The container list is sorted by cpu usage. The most consuming container is on top. ``` ### Parameters None. ## Examples ### Stats `$ ./check_docker_info` returns ```txt OK: Docker stats ID CPU % MEM USAGE / LIMIT MEM % NET I/O PIDs Name ------------------------------------------------------------------------------------------ 7cd2e62febe5 0.00% 15.79MiB / 128MiB 12.34% 673kB / 13.9MB 4 portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn b1733b4d1008 1.07% 40.52MiB / 128MiB 31.66% 915kB / 25.9MB 11 portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt ad4d32e29aef 0.01% 98.01MiB / 3.828GiB 2.50% 104MB / 4.71MB 8 portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6 99a82bf29c82 0.02% 61.07MiB / 3.828GiB 1.56% 4.82MB / 13.1MB 11 portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p 72023160d7b1 0.00% 2.945MiB / 3.828GiB 0.08% 4.27MB / 7.39MB 3 myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf b87843ce6220 0.00% 2.871MiB / 3.828GiB 0.07% 4.08MB / 2.2MB 3 myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el 77e9f55b1113 0.19% 27.79MiB / 3.828GiB 0.71% 88.2MB / 122MB 6 portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s ``` ### CPU `/check_docker_stats -m cpu` ```txt OK: Docker CPU usage total of all containers: 1.50 percent ID CPUPerc ------------------------------------------------------------------------------------------ b1733b4d1008 1.24% portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt 77e9f55b1113 0.25% portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s 99a82bf29c82 0.01% portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p b87843ce6220 0.00% myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el ad4d32e29aef 0.00% portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6 7cd2e62febe5 0.00% portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn 72023160d7b1 0.00% myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf |cpu-sum=1.50;; ``` ### Memory percent Show most memory consuming in percent container on top. It checks the memory of the container. `/check_docker_stats -m mem` ```txt OK: Docker Memory max usage in most consuming container: 30.38 percent ID MemPerc ------------------------------------------------------------------------------------------ b1733b4d1008 30.38% 38.89MiB / 128MiB portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt 7cd2e62febe5 12.21% 15.63MiB / 128MiB portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn ad4d32e29aef 2.21% 86.67MiB / 3.828GiB portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6 99a82bf29c82 1.56% 61.07MiB / 3.828GiB portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p 77e9f55b1113 0.71% 27.8MiB / 3.828GiB portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s 72023160d7b1 0.08% 2.945MiB / 3.828GiB myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf b87843ce6220 0.07% 2.871MiB / 3.828GiB myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el |mem-max=30.38;;;0;100 ``` ### Memory total `/check_docker_stats -m memtotal` ```txt OK: Docker Memory usage of all containers: 224.977 MB ID MemUsage ------------------------------------------------------------------------------------------ ad4d32e29aef 86.43MiB / 3.828GiB portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6 99a82bf29c82 61.08MiB / 3.828GiB portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p b1733b4d1008 39.26MiB / 128MiB portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt 77e9f55b1113 27.67MiB / 3.828GiB portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s 7cd2e62febe5 15.65MiB / 128MiB portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn 72023160d7b1 2.945MiB / 3.828GiB myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf b87843ce6220 2.871MiB / 3.828GiB myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el |memtotal-sum=235906000.000;; ``` ### Netio `/check_docker_stats -m netio` ```txt OK: Docker Network I/O of all containers: 209.363 MB IN .. 191.180 MB OUT ID NetIO ------------------------------------------------------------------------------------------ ad4d32e29aef 106MB / 4.8MB portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6 77e9f55b1113 88.4MB / 123MB portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s 99a82bf29c82 4.91MB / 13.1MB portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p 72023160d7b1 4.35MB / 7.44MB myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf b87843ce6220 4.16MB / 2.24MB myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el b1733b4d1008 932kB / 26.4MB portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt 7cd2e62febe5 686kB / 14.2MB portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn |netio-1=219533064.32;; netio-2=200466759.68;; ``` ### Blockio `/check_docker_stats -m bliockio` ```txt OK: Docker Network I/O of all containers: 5.922 MB IN .. 95.773 MB OUT ID BlockIO ------------------------------------------------------------------------------------------ b1733b4d1008 2.06MB / 0B portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt 99a82bf29c82 1.99MB / 786kB portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p ad4d32e29aef 1.17MB / 95MB portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6 77e9f55b1113 479kB / 8.19kB portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s 7cd2e62febe5 258kB / 0B portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn b87843ce6220 0B / 8.19kB myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el 72023160d7b1 0B / 8.19kB myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf |blockio-1=6210566.72;; blockio-2=100425290.00;; ``` ## Troubleshooting ### Missing environment ```txt sudo: sorry, you are not allowed to preserve the environment UNKNOWN: No data. Unable to fetch Docker information. ``` Solution: you did not set SETENV in the sudoers config file icingaclient ALL=(ALL) NOPASSWD:**SETENV:** /usr/bin/docker ### Docker not found If you get an error message that docker was not found then it is not reachable in ofe of the directories of $PATH. This can happen especialy in rootless docker instances. Solution: Use the parameter `-p /usr/bin` to define a target. Copy the docker binary there.