Monitoring Docker

This article will help you get the Docker plugin for sd-agent configured and returning metrics

Installing the docker plugin package

Install the docker plugin on Debian/Ubuntu:

sudo apt-get install sd-agent-docker

Install the docker plugin on RHEL/CentOS:

sudo yum install sd-agent-docker

Read more about agent plugins.

Configuring the agent to monitor Docker

Add the sd-agent user to the docker group

usermod -a -G docker sd-agent

Configure /etc/sd-agent/conf.d/docker_daemon.yaml

init_config:
  # docker_root: /
  # timeout: 10

  # tls: False
  # tls_client_cert: /path/to/client-cert.pem
  # tls_client_key: /path/to/client-key.pem
  # tls_cacert: /path/to/ca.pem
  # tls_verify: True

instances:
  - ## Daemon and system configuration
    url: "unix://var/run/docker.sock"
    # collect_container_size: false
    # collect_images_stats: false
    # collect_image_size: false
    # collect_disk_stats: true

Remember that you can uncomment any of the above options if you wish to change them.

Verifying the configuration
Execute info to verify the configuration with the following:

sudo /etc/init.d/sd-agent info 

or

/usr/share/python/sd-agent/agent.py info

If the agent has been configured correctly you'll see an output such as:

docker
-----
  - instance #0 [OK]
  - Collected * metrics

You can also view the metrics returned with the following command:

sudo -u sd-agent /usr/share/python/sd-agent/agent.py check docker_daemon

Configuring graphs

Click the name of your server from the Devices list in your Server Density account then go to the Metrics tab. Click the + Graph button on the right then choose the docker metrics to display the graphs. The metrics will also be available to select when building dashboard graphs.

Monitored metrics

Metric	Values
docker.container.size_rootfs	/ Type: float
docker.container.size_rootfs.95percentile 95th percentile of docker.container.size_rootfs	byte / None Type: float
docker.container.size_rootfs.avg Average value of docker.container.size_rootfs	byte / None Type: float
docker.container.size_rootfs.count The rate that the value of docker.container.size_rw was sampled	sample / second Type: float
docker.container.size_rootfs.max Max value of docker.container.size_rootfs	byte / None Type: float
docker.container.size_rootfs.median Median value of docker.container.size_rootfs	byte / None Type: float
docker.container.size_rw	/ Type: float
docker.container.size_rw.95percentile 95th percentile of docker.container.size_rw	byte / None Type: float
docker.container.size_rw.avg Average value of docker.container.size_rw	byte / None Type: float
docker.container.size_rw.count The rate that the value of docker.container.size_rw was sampled	sample / second Type: float
docker.container.size_rw.max Max value of docker.container.size_rw	byte / None Type: float
docker.container.size_rw.median Median value of docker.container.size_rw	byte / None Type: float
docker.containers.running	/ Type: float
docker.containers.stopped	/ Type: float
docker.cpu.system	/ Type: float
docker.cpu.system.95percentile 95th percentile of docker.cpu.system	percent / None Type: float
docker.cpu.system.avg Average value of docker.cpu.system	percent / None Type: float
docker.cpu.system.count The rate that the value of docker.cpu.system was sampled	sample / second Type: float
docker.cpu.system.max Max value of docker.cpu.system	percent / None Type: float
docker.cpu.system.median Median value of docker.cpu.system	percent / None Type: float
docker.cpu.throttled Number of times the cgroup has been throttled	None / None Type: float
docker.cpu.usage The percent of CPU time obtained by this container	percent / None Type: float
docker.cpu.user	/ Type: float
docker.cpu.user.95percentile 95th percentile of docker.cpu.user	percent / None Type: float
docker.cpu.user.avg Average value of docker.cpu.user	percent / None Type: float
docker.cpu.user.count The rate that the value of docker.cpu.user was sampled	sample / second Type: float
docker.cpu.user.max Max value of docker.cpu.user	percent / None Type: float
docker.cpu.user.median Median value of docker.cpu.user	percent / None Type: float
docker.data.free Storage pool disk space free	byte / None Type: float
docker.data.percent The percent of storage pool used	percent / None Type: float
docker.data.total Storage pool disk space total	byte / None Type: float
docker.data.used Storage pool disk space used	byte / None Type: float
docker.image.size Size of all layers of the image on disk	byte / None Type: float
docker.image.virtual_size Size of all layers of the image on disk	byte / None Type: float
docker.images.available The number of top-level images	None / None Type: float
docker.images.intermediate The number of intermediate images, which are intermediate layers that make up other images	None / None Type: float
docker.io.read_bytes	/ Type: float
docker.io.read_bytes.95percentile 95th percentile of docker.io.read_bytes	byte / second Type: float
docker.io.read_bytes.avg Average value of docker.io.read_bytes	byte / second Type: float
docker.io.read_bytes.count The rate that the value of docker.io.read_bytes was sampled	sample / second Type: float
docker.io.read_bytes.max Max value of docker.container.io.read_bytes	byte / second Type: float
docker.io.read_bytes.median Median value of docker.container.io.read_bytes	byte / second Type: float
docker.io.write_bytes	/ Type: float
docker.io.write_bytes.95percentile 95th percentile of docker.io.write_bytes	byte / second Type: float
docker.io.write_bytes.avg Average value of docker.io.write_bytes	byte / second Type: float
docker.io.write_bytes.count The rate that the value of docker.io.write_bytes was sampled	sample / second Type: float
docker.io.write_bytes.max Max value of docker.container.io.write_bytes	byte / second Type: float
docker.io.write_bytes.median Median value of docker.container.io.write_bytes	byte / second Type: float
docker.mem.cache	/ Type: float
docker.mem.cache.95percentile 95th percentile value of docker.mem.cache	byte / None Type: float
docker.mem.cache.avg Average value of docker.mem.cache	byte / None Type: float
docker.mem.cache.count The rate that the value of docker.mem.cache was sampled	sample / second Type: float
docker.mem.cache.max Max value of docker.mem.cache	byte / None Type: float
docker.mem.cache.median Median value of docker.mem.cache	byte / None Type: float
docker.mem.in_use The fraction of used memory to available memory, if the limit is set	fraction / None Type: float
docker.mem.in_use.95percentile 95th percentile of docker.mem.in_use	fraction / None Type: float
docker.mem.in_use.avg Average value of docker.mem.in_use	fraction / None Type: float
docker.mem.in_use.count The rate that the value of docker.mem.in_use was sampled	sample / second Type: float
docker.mem.in_use.max Max value of docker.container.mem.in_use	fraction / None Type: float
docker.mem.in_use.median Median value of docker.container.mem.in_use	fraction / None Type: float
docker.mem.limit The memory limit for the container, if set	byte / None Type: float
docker.mem.limit.95percentile 95th percentile of docker.mem.limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.limit.avg Average value of docker.mem.limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.limit.count The rate that the value of docker.mem.limit was sampled	sample / second Type: float
docker.mem.limit.max Max value of docker.mem.limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.limit.median Median value of docker.mem.limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.rss	/ Type: float
docker.mem.rss.95percentile 95th percentile value of docker.mem.rss	byte / None Type: float
docker.mem.rss.avg Average value of docker.mem.rss	byte / None Type: float
docker.mem.rss.count The rate that the value of docker.mem.rss was sampled	sample / second Type: float
docker.mem.rss.max Max value of docker.mem.rss	byte / None Type: float
docker.mem.rss.median Median value of docker.mem.rss	byte / None Type: float
docker.mem.sw_in_use The fraction of used swap + memory to available swap + memory, if the limit is set	fraction / None Type: float
docker.mem.sw_in_use.95percentile 95th percentile of docker.mem.sw_in_use	fraction / None Type: float
docker.mem.sw_in_use.avg Average value of docker.mem.sw_in_use	fraction / None Type: float
docker.mem.sw_in_use.count The rate that the value of docker.mem.sw_in_use was sampled	sample / second Type: float
docker.mem.sw_in_use.max Max value of docker.container.mem.sw_in_use	fraction / None Type: float
docker.mem.sw_in_use.median Median value of docker.container.mem.sw_in_use	fraction / None Type: float
docker.mem.sw_limit The swap + memory limit for the container, if set	byte / None Type: float
docker.mem.sw_limit.95percentile 95th percentile of docker.mem.sw_limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.sw_limit.avg Average value of docker.mem.sw_limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.sw_limit.count The rate that the value of docker.mem.sw_limit was sampled	sample / second Type: float
docker.mem.sw_limit.max Max value of docker.mem.sw_limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.sw_limit.median Median value of docker.mem.sw_limit. Ordinarily this value will not change	byte / None Type: float
docker.mem.swap	/ Type: float
docker.mem.swap.95percentile 95th percentile value of docker.mem.swap	byte / None Type: float
docker.mem.swap.avg Average value of docker.mem.swap	byte / None Type: float
docker.mem.swap.count The rate that the value of docker.mem.swap was sampled	sample / second Type: float
docker.mem.swap.max Max value of docker.mem.swap	byte / None Type: float
docker.mem.swap.median Median value of docker.mem.swap	byte / None Type: float
docker.metadata.free Storage pool metadata space free	byte / None Type: float
docker.metadata.percent The percent of storage pool metadata used	percent / None Type: float
docker.metadata.total Storage pool metadata space total	byte / None Type: float
docker.metadata.used Storage pool metadata space used	byte / None Type: float
docker.net.bytes_rcvd	/ Type: float
docker.net.bytes_rcvd.95percentile 95th percentile of docker.net.bytes_rcvd	byte / second Type: float
docker.net.bytes_rcvd.avg Average value of docker.net.bytes_rcvd	byte / second Type: float
docker.net.bytes_rcvd.count The rate that the value of docker.net.bytes_rcvd was sampled	sample / second Type: float
docker.net.bytes_rcvd.max Max value of docker.container.net.bytes_rcvd	byte / second Type: float
docker.net.bytes_rcvd.median Median value of docker.container.net.bytes_rcvd	byte / second Type: float
docker.net.bytes_sent	/ Type: float
docker.net.bytes_sent_bytes.95percentile 95th percentile of docker.net.bytes_sent_bytes	byte / second Type: float
docker.net.bytes_sent_bytes.avg Average value of docker.net.bytes_sent_bytes	byte / second Type: float
docker.net.bytes_sent_bytes.count The rate that the value of docker.net.bytes_sent_bytes was sampled	sample / second Type: float
docker.net.bytes_sent_bytes.max Max value of docker.container.net.bytes_sent_bytes	byte / second Type: float
docker.net.bytes_sent_bytes.median Median value of docker.container.net.bytes_sent_bytes	byte / second Type: float
docker.virtual.size	/ Type: float

Metrics Conversion

Please refer to the metrics conversion document to map old metrics names to the new metric names.