Skip to content
Snippets Groups Projects
Select Git revision
  • 4cf050d26c745c80ce17d03a4d3973897ecb9995
  • master default protected
  • simple-task/7248-eol-check-add-node-22
  • 6877_check_iml_deployment
4 results

check_docker_stats.md

Blame
  • CHECK_DOCKER_STATS

    Introduction

    check_docker_stats shows the docker resources of containers. This check sends performance data.

    Requirements

    • docker Docker must be installed
    • jq must be installed - commandline JSON processor
    • bc must be installed - an arbitrary precision calculator language
    • sudo permissions on docker command
    icingaclient ALL=(ALL) NOPASSWD:SETENV: /usr/bin/docker

    Includes

    Additional needed files in the current folder that this check can run:

    • inc_dockerfunctions.sh
    • inc_pluginfunctions

    Rootless docker

    It works with docker setups as root and can handle rootless docker instances.

    To bring it up and running with an unpriviledged icinga user it must be able to access docker. Copy the docker binary eg. from /home/dockeruser/bin/docker to /usr/bin/. Then use the parameter -p /usr/bin.

    If DOCKER_HOST is not set then the docker socket is detected from a running docker instance and will be fetched from the process list.

    Syntax

    ______________________________________________________________________
    
    CHECK_DOCKER_STATS
    v1.1
    
    (c) Institute for Medical Education - University of Bern
    Licence: GNU GPL 3
    
    https://os-docs.iml.unibe.ch/icinga-checks/Checks/check_docker_stats.html
    ______________________________________________________________________
    
    Show resources of running docker instances.
    
    This check provides performance data except on mode "all".
    
    USAGE:
      check_docker_stats [OPTIONS]
    
    OPTIONS:
    
      General:
      -h, --help        this help
      -d, --debug       Debug; Flag: show docker env values.
    
      Connect to docker:
      -p, --path        Custom directory for docker binary
      -t, --target      Custom docker target; value for DOCKER_HOST
                        Needed only if Docker does not run on a unix socket or
                        multiple users run a rootless docker daemon.
    
      mode:
      -m, --mode MODE   Mode what kind of information to show.
                        The output of container list is sorted by the maximum 
                        value first.
    
                          all       default; This view is unsorted.
                                    This check has no performance data.
    
                          cpu       CPU usage total of all containers
                          mem       Memory max usage of most consuming container
    
                          memtotal  sum of memory usage
                          netio     sum of netio of all containers IN and OUT
                          blockio   sum of blockio of all containers IN and OUT
    
    EXAMPLES:
      check_docker_stats
        Show stats of all containers
    
      check_docker_stats -p /usr/bin
        Show stats of all containers. The docker binary will be searched in
        given path first - then in all other dirs of $PATH
    
      check_docker_stats -d
        Show stats of all containers and json with docker env infos.
    
      check_docker_stats -m cpu
        Show cpu usage total of all containers. The container list is sorted
        by cpu usage. The most consuming container is on top.
    

    Parameters

    None.

    Examples

    Stats

    $ ./check_docker_info returns

    OK: Docker stats
    
    ID              CPU %   MEM USAGE / LIMIT       MEM %   NET I/O         PIDs    Name
    ------------------------------------------------------------------------------------------
    7cd2e62febe5    0.00%   15.79MiB / 128MiB       12.34%  673kB / 13.9MB  4       portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn
    b1733b4d1008    1.07%   40.52MiB / 128MiB       31.66%  915kB / 25.9MB  11      portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt
    ad4d32e29aef    0.01%   98.01MiB / 3.828GiB     2.50%   104MB / 4.71MB  8       portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6
    99a82bf29c82    0.02%   61.07MiB / 3.828GiB     1.56%   4.82MB / 13.1MB 11      portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p
    72023160d7b1    0.00%   2.945MiB / 3.828GiB     0.08%   4.27MB / 7.39MB 3       myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf
    b87843ce6220    0.00%   2.871MiB / 3.828GiB     0.07%   4.08MB / 2.2MB  3       myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el
    77e9f55b1113    0.19%   27.79MiB / 3.828GiB     0.71%   88.2MB / 122MB  6       portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s

    CPU

    /check_docker_stats -m cpu

    OK: Docker CPU usage total of all containers: 1.50 percent
    
    ID              CPUPerc
    ------------------------------------------------------------------------------------------
    b1733b4d1008    1.24%   portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt
    77e9f55b1113    0.25%   portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s
    99a82bf29c82    0.01%   portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p
    b87843ce6220    0.00%   myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el
    ad4d32e29aef    0.00%   portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6
    7cd2e62febe5    0.00%   portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn
    72023160d7b1    0.00%   myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf
    
     |cpu-sum=1.50;;

    Memory percent

    Show most memory consuming in percent container on top. It checks the memory of the container.

    /check_docker_stats -m mem

    OK: Docker Memory max usage in most consuming container: 30.38 percent
    
    ID              MemPerc
    ------------------------------------------------------------------------------------------
    b1733b4d1008    30.38%  38.89MiB / 128MiB       portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt
    7cd2e62febe5    12.21%  15.63MiB / 128MiB       portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn
    ad4d32e29aef    2.21%   86.67MiB / 3.828GiB     portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6
    99a82bf29c82    1.56%   61.07MiB / 3.828GiB     portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p
    77e9f55b1113    0.71%   27.8MiB / 3.828GiB      portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s
    72023160d7b1    0.08%   2.945MiB / 3.828GiB     myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf
    b87843ce6220    0.07%   2.871MiB / 3.828GiB     myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el
    
     |mem-max=30.38;;;0;100 

    Memory total

    /check_docker_stats -m memtotal

    OK: Docker Memory usage of all containers: 224.977 MB
    
    ID              MemUsage
    ------------------------------------------------------------------------------------------
    ad4d32e29aef    86.43MiB / 3.828GiB     portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6
    99a82bf29c82    61.08MiB / 3.828GiB     portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p
    b1733b4d1008    39.26MiB / 128MiB       portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt
    77e9f55b1113    27.67MiB / 3.828GiB     portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s
    7cd2e62febe5    15.65MiB / 128MiB       portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn
    72023160d7b1    2.945MiB / 3.828GiB     myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf
    b87843ce6220    2.871MiB / 3.828GiB     myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el
    
     |memtotal-sum=235906000.000;;

    Netio

    /check_docker_stats -m netio

    OK: Docker Network I/O of all containers: 209.363 MB IN .. 191.180 MB OUT
    
    ID              NetIO
    ------------------------------------------------------------------------------------------
    ad4d32e29aef    106MB / 4.8MB   portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6
    77e9f55b1113    88.4MB / 123MB  portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s
    99a82bf29c82    4.91MB / 13.1MB portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p
    72023160d7b1    4.35MB / 7.44MB myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf
    b87843ce6220    4.16MB / 2.24MB myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el
    b1733b4d1008    932kB / 26.4MB  portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt
    7cd2e62febe5    686kB / 14.2MB  portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn
    
     |netio-1=219533064.32;; netio-2=200466759.68;;

    Blockio

    /check_docker_stats -m bliockio

    OK: Docker Network I/O of all containers: 5.922 MB IN .. 95.773 MB OUT
    
    ID              BlockIO
    ------------------------------------------------------------------------------------------
    b1733b4d1008    2.06MB / 0B     portainer_monitoring_cadvisor.qm9ur2pzl7ngw2o6wb2s7fx4i.vel7o1qcd2ezit82ep7r5qqlt
    99a82bf29c82    1.99MB / 786kB  portainer_monitoring_grafana.1.1fux1t4q7jy7ja5szun3hbq3p
    ad4d32e29aef    1.17MB / 95MB   portainer_monitoring_prometheus.1.rlwvmh5hbo13sujt1rd5szqr6
    77e9f55b1113    479kB / 8.19kB  portainer_agent.qm9ur2pzl7ngw2o6wb2s7fx4i.i7cohx1xim0ljmlpaeamd2k9s
    7cd2e62febe5    258kB / 0B      portainer_monitoring_node-exporter.qm9ur2pzl7ngw2o6wb2s7fx4i.d6txef4ivx5wizt7ixg7z6rsn
    b87843ce6220    0B / 8.19kB     myapp_author.1.0rsdcjt8tjokcvobf9jgqk0el
    72023160d7b1    0B / 8.19kB     myapp_viewer.1.76pvzu0srf13nx6vcd99kqjhf
    
     |blockio-1=6210566.72;; blockio-2=100425290.00;;

    Troubleshooting

    Missing environment

    sudo: sorry, you are not allowed to preserve the environment
    UNKNOWN: No data. Unable to fetch Docker information.

    Solution: you did not set SETENV in the sudoers config file

    icingaclient ALL=(ALL) NOPASSWD:SETENV: /usr/bin/docker

    Docker not found

    If you get an error message that docker was not found then it is not reachable in ofe of the directories of $PATH. This can happen especialy in rootless docker instances.

    Solution: Use the parameter -p /usr/bin to define a target. Copy the docker binary there.