Monitoring Kubernetes State

This article will help you get the Kubernetes_state plugin for sd-agent configured and returning metrics

Installing the kubernetes_state plugin package

Install the kubernetes_state plugin on Debian/Ubuntu:

sudo apt-get install sd-agent-kubernetes_state

Install the kubernetes_state plugin on RHEL/CentOS:

sudo yum install sd-agent-kubernetes_state

Read more about agent plugins.

Verifying the configuration
Execute info to verify the configuration with the following:

sudo /etc/init.d/sd-agent info 

or

/usr/share/python/sd-agent/agent.py info

If the agent has been configured correctly you'll see an output such as:

kubernetes_state
-----
  - instance #0 [OK]
  - Collected * metrics

You can also view the metrics returned with the following command:

sudo -u sd-agent /usr/share/python/sd-agent/agent.py check kubernetes_state

Configuring graphs

Click the name of your server from the Devices list in your Server Density account then go to the Metrics tab. Click the + Graph button on the right then choose the kubernetes_state metrics to display the graphs. The metrics will also be available to select when building dashboard graphs.

Screen_Shot_2018-01-18_at_11.46.12.png

Monitored metrics

MetricValues
kubernetes_state.container.cpu_limit

The limit on cpu cores to be used by a container
cpu / None
Type: float
kubernetes_state.container.cpu_requested

The number of requested cpu cores by a container
cpu / None
Type: float
kubernetes_state.container.memory_limit

The limit on memory to be used by a container
byte / None
Type: float
kubernetes_state.container.memory_requested

The number of requested memory bytes by a container
byte / None
Type: float
kubernetes_state.container.ready

Whether the containers readiness check succeeded
None / None
Type: float
kubernetes_state.container.restarts

The number of restarts per container
None / None
Type: float
kubernetes_state.container.running

Whether the container is currently in running state
None / None
Type: float
kubernetes_state.container.terminated

Whether the container is currently in terminated state
None / None
Type: float
kubernetes_state.container.waiting

Whether the container is currently in waiting state
None / None
Type: float
kubernetes_state.daemonset.desired

The number of nodes that should be running the daemon pod
None / None
Type: float
kubernetes_state.daemonset.misscheduled

The number of nodes running a daemon pod but are not supposed to
None / None
Type: float
kubernetes_state.daemonset.ready

The number of nodes that should be running the daemon pod and have one or more running and ready
None / None
Type: float
kubernetes_state.daemonset.scheduled

The number of nodes running at least one daemon pod and that are supposed to
None / None
Type: float
kubernetes_state.deployment.paused

Whether a deployment is paused
None / None
Type: float
kubernetes_state.deployment.replicas

The number of replicas per deployment
None / None
Type: float
kubernetes_state.deployment.replicas_available

The number of available replicas per deployment
None / None
Type: float
kubernetes_state.deployment.replicas_desired

The number of desired replicas per deployment
None / None
Type: float
kubernetes_state.deployment.replicas_unavailable

The number of unavailable replicas per deployment
None / None
Type: float
kubernetes_state.deployment.replicas_updated

The number of updated replicas per deployment
None / None
Type: float
kubernetes_state.deployment.rollingupdate.max_unavailable

Maximum number of unavailable replicas during a rolling update
None / None
Type: float
kubernetes_state.limitrange.cpu.default

Default CPU limit if not specified
None / None
Type: float
kubernetes_state.limitrange.cpu.default_request

Default CPU request if not specified
None / None
Type: float
kubernetes_state.limitrange.cpu.max

Maximum CPU limit for this type
None / None
Type: float
kubernetes_state.limitrange.cpu.max_limit_request_ratio

Maximum CPU limit / request ratio
None / None
Type: float
kubernetes_state.limitrange.cpu.min

Minimum CPU request for this type
None / None
Type: float
kubernetes_state.limitrange.memory.default

Default memory limit if not specified
None / None
Type: float
kubernetes_state.limitrange.memory.default_request

Default memory request if not specified
None / None
Type: float
kubernetes_state.limitrange.memory.max

Maximum memory limit for this type
None / None
Type: float
kubernetes_state.limitrange.memory.max_limit_request_ratio

Maximum memory limit / request ratio
None / None
Type: float
kubernetes_state.limitrange.memory.min

Minimum memory request for this type
None / None
Type: float
kubernetes_state.node.cpu_allocatable

The CPU resources of a node that are available for scheduling
cpu / None
Type: float
kubernetes_state.node.cpu_capacity

The total CPU resources of the node
cpu / None
Type: float
kubernetes_state.node.memory_allocatable

The memory resources of a node that are available for scheduling
byte / None
Type: float
kubernetes_state.node.memory_capacity

The total memory resources of the node
byte / None
Type: float
kubernetes_state.node.pods_allocatable

The pod resources of a node that are available for scheduling
None / None
Type: float
kubernetes_state.node.pods_capacity

The total pod resources of the node
None / None
Type: float
kubernetes_state.node.status

The number of nodes in a given status per tag
None / None
Type: float
kubernetes_state.pod.ready

Whether the pod is ready to serve requests
None / None
Type: float
kubernetes_state.pod.scheduled

Reports the status of the scheduling process for the pod with its tags
None / None
Type: float
kubernetes_state.replicaset.fully_labeled_replicas

The number of fully labeled replicas per ReplicaSet
None / None
Type: float
kubernetes_state.replicaset.replicas

The number of replicas per ReplicaSet
None / None
Type: float
kubernetes_state.replicaset.replicas_desired

Number of desired pods for a ReplicaSet
None / None
Type: float
kubernetes_state.replicaset.replicas_ready

The number of ready replicas per ReplicaSet
None / None
Type: float
kubernetes_state.replicationcontroller.fully_labeled_replicas

The number of fully labeled replicas per ReplicationController
None / None
Type: float
kubernetes_state.replicationcontroller.replicas

The number of replicas per ReplicationController
None / None
Type: float
kubernetes_state.replicationcontroller.replicas_available

The number of available replicas per ReplicationController
None / None
Type: float
kubernetes_state.replicationcontroller.replicas_desired

Number of desired replicas for a ReplicationController
None / None
Type: float
kubernetes_state.replicationcontroller.replicas_ready

The number of ready replicas per ReplicationController
None / None
Type: float
kubernetes_state.resourcequota.limits.cpu.limit

Hard limit on the sum of CPU core limits for a resource quota
cpu / None
Type: float
kubernetes_state.resourcequota.limits.cpu.used

Observed sum of limits for CPU cores for a resource quota
cpu / None
Type: float
kubernetes_state.resourcequota.limits.memory.limit

Hard limit on the sum of memory bytes limits for a resource quota
byte / None
Type: float
kubernetes_state.resourcequota.limits.memory.used

Observed sum of limits for memory bytes for a resource quota
byte / None
Type: float
kubernetes_state.resourcequota.persistentvolumeclaims.limit

Hard limit of the number of PVC for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.persistentvolumeclaims.used

Observed number of persistent volume claims used for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.pods.limit

Hard limit of the number of pods for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.pods.used

Observed number of pods used for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.requests.cpu.limit

Hard limit on the total of CPU core requested for a resource quota
cpu / None
Type: float
kubernetes_state.resourcequota.requests.cpu.used

Observed sum of CPU cores requested for a resource quota
cpu / None
Type: float
kubernetes_state.resourcequota.requests.memory.limit

Hard limit on the total of memory bytes requested for a resource quota
byte / None
Type: float
kubernetes_state.resourcequota.requests.memory.used

Observed sum of memory bytes requested for a resource quota
byte / None
Type: float
kubernetes_state.resourcequota.requests.storage.limit

Hard limit on the total of storage bytes requested for a resource quota
byte / None
Type: float
kubernetes_state.resourcequota.requests.storage.used

Observed sum of storage bytes requested for a resource quota
byte / None
Type: float
kubernetes_state.resourcequota.services.limit

Hard limit of the number of services for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.services.loadbalancers.limit

Hard limit of the number of loadbalancers for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.services.loadbalancers.used

Observed number of loadbalancers used for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.services.nodeports.limit

Hard limit of the number of node ports for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.services.nodeports.used

Observed number of node ports used for a resource quota
None / None
Type: float
kubernetes_state.resourcequota.services.used

Observed number of services used for a resource quota
None / None
Type: float
kubernetes_state.statefulset.replicas

The number of replicas per statefulset
None / None
Type: float
kubernetes_state.statefulset.replicas_desired

The number of desired replicas per statefulset
None / None
Type: float
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Monday  —  Friday.

10am  —  6pm UK.

Dedicated Support.