Monitoring Kubernetes DNS

This article will help you get the Kube_dns plugin for sd-agent configured and returning metrics

Installing the kube_dns plugin package

Install the kube_dns plugin on Debian/Ubuntu:

sudo apt-get install sd-agent-kube-dns

Install the kube_dns plugin on RHEL/CentOS:

sudo yum install sd-agent-kube-dns

Read more about agent plugins.

Configuring the agent to monitor Kubernetes DNS

1. Configure /etc/sd-agent/conf.d/kube_dns.yaml

init_config:

instances:
    # url of the metrics endpoint of prometheus
  - prometheus_endpoint: http://localhost:10055/metrics
    # The histogram buckets can be noisy and generate a lot of tags.
    # send_histograms_buckets controls whether or not you want to pull them.
    #
    # send_histograms_buckets: True
    #
    # Note that like all checks based on the PrometheusCheck class, you can add
    # tags to the instance that will be added to all the metrics of this check
    # instance.
    #
    # tags:
    #   - 'mytag1:myValue1'

Ensure that the options correctly point to your server and port.

2. Restart the agent

sudo /etc/init.d/sd-agent restart

or

sudo systemctl restart sd-agent

Verifying the configuration
Execute info to verify the configuration with the following:

sudo /etc/init.d/sd-agent info 

or

/usr/share/python/sd-agent/agent.py info

If the agent has been configured correctly you'll see an output such as:

kube_dns
-----
  - instance #0 [OK]
  - Collected * metrics

You can also view the metrics returned with the following command:

sudo -u sd-agent /usr/share/python/sd-agent/agent.py check kube_dns

Configuring graphs

Click the name of your server from the Devices list in your Server Density account then go to the Metrics tab. Click the + Graph button on the right then choose the kube_dns metrics to display the graphs. The metrics will also be available to select when building dashboard graphs.

Monitored metrics

Metric	Values
kubedns.cachemiss_count Number of DNS requests that result in a cache miss.	request / None Type: float
kubedns.error_count Number of DNS requests resulting in an error.	error / None Type: float
kubedns.request_count Number of DNS requests made.	request / None Type: float
kubedns.request_duration.seconds.count Number of requests on which the kubedns.request_duration.seconds.sum metric is evaluated.	request / None Type: float
kubedns.request_duration.seconds.sum Time (in seconds) each request took to resolve.	second / second Type: float
kubedns.response_size.bytes.count Number of responses on which the kubedns.response_size.bytes.sum metric is evaluated.	response / None Type: float
kubedns.response_size.bytes.sum Size of the returns response in bytes.	byte / None Type: float