Monitoring Elastic Search

This article will help you get the Elasticsearch plugin for sd-agent configured and returning metrics

Installing the elasticsearch plugin package

Install the elasticsearch plugin on Debian/Ubuntu:

sudo apt-get install sd-agent-elastic

Install the elasticsearch plugin on RHEL/CentOS:

sudo yum install sd-agent-elastic

Read more about agent plugins.

Configuring the agent to monitor Elastic

1. Configure /etc/sd-agent/conf.d/elastic.yaml

init_config:

instances:
- url: http://localhost:9200
  • If Elastic requires authentication or you wish to specify other connection details, amend the rest of the config file as necessary

2. Restart the agent

sudo /etc/init.d/sd-agent restart

or

sudo systemctl restart sd-agent

Verifying the configuration
Execute info to verify the configuration with the following:

sudo /etc/init.d/sd-agent info 

or

/usr/share/python/sd-agent/agent.py info

If the agent has been configured correctly you'll see an output such as:

elasticsearch
-----
  - instance #0 [OK]
  - Collected * metrics

You can also view the metrics returned with the following command:

sudo -u sd-agent /usr/share/python/sd-agent/agent.py check elastic

Configuring graphs

Click the name of your server from the Devices list in your Server Density account then go to the Metrics tab. Click the + Graph button on the right then choose the elasticsearch metrics to display the graphs. The metrics will also be available to select when building dashboard graphs.

Screen_Shot_2018-01-18_at_11.46.12.png

Monitored metrics

MetricValues
elasticsearch.active_primary_shards

/
Type: float
elasticsearch.active_shards

/
Type: float
elasticsearch.breakers.fielddata.estimated_size_in_bytes

The estimated size in bytes of the field data circuit breaker.
byte / None
Type: float
elasticsearch.breakers.fielddata.overhead

The constant multiplier for byte estimations of the field data circuit breaker.
None / None
Type: float
elasticsearch.breakers.fielddata.tripped

The number of times the field data circuit breaker has tripped.
None / None
Type: float
elasticsearch.breakers.parent.estimated_size_in_bytes

The estimated size in bytes of the parent circuit breaker.
byte / None
Type: float
elasticsearch.breakers.parent.overhead

The constant multiplier for byte estimations of the parent circuit breaker.
None / None
Type: float
elasticsearch.breakers.parent.tripped

The number of times the parent circuit breaker has tripped.
None / None
Type: float
elasticsearch.breakers.request.estimated_size_in_bytes

The estimated size in bytes of the request circuit breaker.
byte / None
Type: float
elasticsearch.breakers.request.overhead

The constant multiplier for byte estimations of the request circuit breaker.
None / None
Type: float
elasticsearch.breakers.request.tripped

The number of times the request circuit breaker has tripped.
None / None
Type: float
elasticsearch.cache.field.evictions

The total number of evictions from the field data cache.
eviction / None
Type: float
elasticsearch.cache.field.size

The size of the field cache.
byte / None
Type: float
elasticsearch.cache.filter.count

The number of items in the filter cache.
item / None
Type: float
elasticsearch.cache.filter.evictions

The total number of evictions from the filter cache.
eviction / None
Type: float
elasticsearch.cache.filter.size

The size of the filter cache.
byte / None
Type: float
elasticsearch.cluster_status

/
Type: float
elasticsearch.docs.count

/
Type: float
elasticsearch.docs.deleted

/
Type: float
elasticsearch.fielddata.evictions

/
Type: float
elasticsearch.fielddata.size

byte /
Type: float
elasticsearch.flush.total

/
Type: float
elasticsearch.flush.total.time

/
Type: float
elasticsearch.fs.total.available_in_bytes

byte /
Type: float
elasticsearch.fs.total.disk_io_op

The total I/O operations on the file store.
operation / None
Type: float
elasticsearch.fs.total.disk_io_size_in_bytes

Total bytes used for all I/O operations on the file store.
byte / None
Type: float
elasticsearch.fs.total.disk_read_size_in_bytes

The total bytes read from the file store.
byte / None
Type: float
elasticsearch.fs.total.disk_reads

The total number of reads from the file store.
read / None
Type: float
elasticsearch.fs.total.disk_write_size_in_bytes

The total bytes written to the file store.
byte / None
Type: float
elasticsearch.fs.total.disk_writes

The total number of writes to the file store.
write / None
Type: float
elasticsearch.fs.total.free_in_bytes

byte /
Type: float
elasticsearch.fs.total.total_in_bytes

byte /
Type: float
elasticsearch.get.active

/
Type: float
elasticsearch.get.current

/
Type: float
elasticsearch.get.exists.time

second /
Type: float
elasticsearch.get.exists.total

/
Type: float
elasticsearch.get.missing.time

second /
Type: float
elasticsearch.get.missing.total

/
Type: float
elasticsearch.get.queue

/
Type: float
elasticsearch.get.threads

/
Type: float
elasticsearch.get.time

second /
Type: float
elasticsearch.get.total

/
Type: float
elasticsearch.http.current_open

/
Type: float
elasticsearch.http.total_opened

/
Type: float
elasticsearch.id_cache.size

The size of the id cache
byte / None
Type: float
elasticsearch.indexing.delete.current

/
Type: float
elasticsearch.indexing.delete.time

second /
Type: float
elasticsearch.indexing.delete.total

/
Type: float
elasticsearch.indexing.index.current

/
Type: float
elasticsearch.indexing.index.time

second /
Type: float
elasticsearch.indexing.index.total

/
Type: float
elasticsearch.indices.indexing.index_failed

The number of failed indexing operations.
None / None
Type: float
elasticsearch.indices.indexing.throttle_time

The total time indexing waited due to throttling.
millisecond / None
Type: float
elasticsearch.indices.query_cache.evictions

The number of query cache evictions.
eviction / None
Type: float
elasticsearch.indices.query_cache.hit_count

The number of query cache hits.
hit / None
Type: float
elasticsearch.indices.query_cache.memory_size_in_bytes

The memory used by the query cache.
byte / None
Type: float
elasticsearch.indices.query_cache.miss_count

The number of query cache misses.
miss / None
Type: float
elasticsearch.indices.recovery.current_as_source

The number of ongoing recoveries for which a shard serves as a source.
None / None
Type: float
elasticsearch.indices.recovery.current_as_target

The number of ongoing recoveries for which a shard serves as a target.
None / None
Type: float
elasticsearch.indices.recovery.throttle_time

The total time recoveries waited due to throttling.
millisecond / None
Type: float
elasticsearch.indices.request_cache.evictions

The number of request cache evictions.
eviction / None
Type: float
elasticsearch.indices.request_cache.hit_count

The number of request cache hits.
hit / None
Type: float
elasticsearch.indices.request_cache.memory_size_in_bytes

The memory used by the request cache.
byte / None
Type: float
elasticsearch.indices.request_cache.miss_count

The number of request cache misses.
miss / None
Type: float
elasticsearch.indices.segments.count

/
Type: float
elasticsearch.indices.segments.doc_values_memory_in_bytes

The memory used by doc values.
byte / None
Type: float
elasticsearch.indices.segments.fixed_bit_set_memory_in_bytes

/
Type: float
elasticsearch.indices.segments.index_writer_max_memory_in_bytes

/
Type: float
elasticsearch.indices.segments.index_writer_memory_in_bytes

/
Type: float
elasticsearch.indices.segments.memory_in_bytes

byte /
Type: float
elasticsearch.indices.segments.norms_memory_in_bytes

The memory used by norms.
byte / None
Type: float
elasticsearch.indices.segments.stored_fields_memory_in_bytes

The memory used by stored fields.
byte / None
Type: float
elasticsearch.indices.segments.term_vectors_memory_in_bytes

The memory used by term vectors.
byte / None
Type: float
elasticsearch.indices.segments.terms_memory_in_bytes

The memory used by terms.
byte / None
Type: float
elasticsearch.indices.segments.version_map_memory_in_bytes

byte /
Type: float
elasticsearch.indices.translog.operations

/
Type: float
elasticsearch.indices.translog.size_in_bytes

byte /
Type: float
elasticsearch.initializing_shards

/
Type: float
elasticsearch.merges.current

/
Type: float
elasticsearch.merges.current.docs

/
Type: float
elasticsearch.merges.current.size

byte /
Type: float
elasticsearch.merges.total

/
Type: float
elasticsearch.merges.total.docs

/
Type: float
elasticsearch.merges.total.size

/
Type: float
elasticsearch.merges.total.time

second /
Type: float
elasticsearch.number_of_data_nodes

/
Type: float
elasticsearch.number_of_nodes

/
Type: float
elasticsearch.pending_tasks_priority_high

/
Type: float
elasticsearch.pending_tasks_priority_urgent

/
Type: float
elasticsearch.pending_tasks_total

/
Type: float
elasticsearch.primaries.docs.count

The total number of documents in the primary shards.
document / None
Type: float
elasticsearch.primaries.docs.deleted

The total number of documents deleted from the primary shards.
document / None
Type: float
elasticsearch.primaries.flush.total

The total number of index flushes to disk from the primary shards since start.
flush / None
Type: float
elasticsearch.primaries.flush.total.time

The total time spent flushing the index to disk from the primary shards.
second / None
Type: float
elasticsearch.primaries.get.current

The number of get requests currently running on the primary shards.
request / None
Type: float
elasticsearch.primaries.get.exists.time

The total time spent on get requests from the primary shards where the document existed.
request / None
Type: float
elasticsearch.primaries.get.exists.total

The total number of get requests on primary shards where the document existed.
request / None
Type: float
elasticsearch.primaries.get.missing.time

The total time spent on get requests from the primary shards where the document was missing.
second / None
Type: float
elasticsearch.primaries.get.missing.total

The total number of get requests from the primary shards where the document was missing.
request / None
Type: float
elasticsearch.primaries.get.time

The total time spent on get requests from the primary shards.
second / None
Type: float
elasticsearch.primaries.get.total

The total number of get requests from the primary shards.
request / None
Type: float
elasticsearch.primaries.indexing.delete.current

The number of documents currently being deleted from an index on the primary shards.
document / None
Type: float
elasticsearch.primaries.indexing.delete.time

The total time spent deleting documents from an index on the primary shards.
second / None
Type: float
elasticsearch.primaries.indexing.delete.total

The total number of documents deleted from an index on the primary shards.
document / None
Type: float
elasticsearch.primaries.indexing.index.current

The number of documents currently being indexed to an index on the primary shards.
document / None
Type: float
elasticsearch.primaries.indexing.index.time

The total time spent indexing documents to an index on the primary shards.
second / None
Type: float
elasticsearch.primaries.indexing.index.total

The total number of documents indexed to an index on the primary shards.
document / None
Type: float
elasticsearch.primaries.merges.current

The number of currently active segment merges on the primary shards.
merge / None
Type: float
elasticsearch.primaries.merges.current.docs

The number of documents across segments currently being merged on the primary shards.
document / None
Type: float
elasticsearch.primaries.merges.current.size

The size of the segments currently being merged on the primary shards.
byte / None
Type: float
elasticsearch.primaries.merges.total

The total number of segment merges on the primary shards.
merge / None
Type: float
elasticsearch.primaries.merges.total.docs

The total number of documents across all merged segments on the primary shards.
document / None
Type: float
elasticsearch.primaries.merges.total.size

The total size of all merged segments on the primary shards.
byte / None
Type: float
elasticsearch.primaries.merges.total.time

The total time spent on segment merging on the primary shards.
second / None
Type: float
elasticsearch.primaries.refresh.total

The total number of index refreshes on the primary shards.
refresh / None
Type: float
elasticsearch.primaries.refresh.total.time

The total time spent on index refreshes on the primary shards.
second / None
Type: float
elasticsearch.primaries.search.fetch.current

The number of query fetches currently running on the primary shards.
fetch / None
Type: float
elasticsearch.primaries.search.fetch.time

The total time spent on query fetches on the primary shards.
second / None
Type: float
elasticsearch.primaries.search.fetch.total

The total number of query fetches on the primary shards.
fetch / None
Type: float
elasticsearch.primaries.search.query.current

The number of currently active queries on the primary shards.
query / None
Type: float
elasticsearch.primaries.search.query.time

The total time spent querying on the primary shards.
second / None
Type: float
elasticsearch.primaries.search.query.total

The total number of queries to the primary shards.
query / None
Type: float
elasticsearch.primaries.store.size

The total size of all the primary shards.
byte / None
Type: float
elasticsearch.process.open_fd

/
Type: float
elasticsearch.refresh.total

/
Type: float
elasticsearch.refresh.total.time

/
Type: float
elasticsearch.relocating_shards

/
Type: float
elasticsearch.search.active

/
Type: float
elasticsearch.search.fetch.current

/
Type: float
elasticsearch.search.fetch.open_contexts

/
Type: float
elasticsearch.search.fetch.time

second /
Type: float
elasticsearch.search.fetch.total

/
Type: float
elasticsearch.search.query.current

/
Type: float
elasticsearch.search.query.time

second /
Type: float
elasticsearch.search.query.total

/
Type: float
elasticsearch.search.queue

/
Type: float
elasticsearch.search.threads

/
Type: float
elasticsearch.store.size

byte /
Type: float
elasticsearch.thread_pool.bulk.active

/
Type: float
elasticsearch.thread_pool.bulk.queue

/
Type: float
elasticsearch.thread_pool.bulk.rejected

/
Type: float
elasticsearch.thread_pool.bulk.threads

/
Type: float
elasticsearch.thread_pool.fetch_shard_started.active

The number of active threads in the fetch shard started pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_started.queue

The number of queued threads in the fetch shard started pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_started.rejected

The number of rejected threads in the fetch shard started pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_started.threads

The total number of threads in the fetch shard started pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_store.active

The number of active threads in the fetch shard store pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_store.queue

The number of queued threads in the fetch shard store pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_store.rejected

The number of rejected threads in the fetch shard store pool.
thread / None
Type: float
elasticsearch.thread_pool.fetch_shard_store.threads

The total number of threads in the fetch shard store pool.
thread / None
Type: float
elasticsearch.thread_pool.flush.active

/
Type: float
elasticsearch.thread_pool.flush.queue

/
Type: float
elasticsearch.thread_pool.flush.rejected

The number of rejected threads in the flush pool.
thread / None
Type: float
elasticsearch.thread_pool.flush.threads

/
Type: float
elasticsearch.thread_pool.force_merge.active

The number of active threads for force merge operations.
thread / None
Type: float
elasticsearch.thread_pool.force_merge.queue

The number of queued threads for force merge operations.
thread / None
Type: float
elasticsearch.thread_pool.force_merge.rejected

The number of rejected threads for force merge operations.
thread / None
Type: float
elasticsearch.thread_pool.force_merge.threads

The total number of threads for force merge operations.
thread / None
Type: float
elasticsearch.thread_pool.generic.active

/
Type: float
elasticsearch.thread_pool.generic.queue

/
Type: float
elasticsearch.thread_pool.generic.rejected

The number of rejected threads in the generic pool.
thread / None
Type: float
elasticsearch.thread_pool.generic.threads

/
Type: float
elasticsearch.thread_pool.get.active

/
Type: float
elasticsearch.thread_pool.get.queue

/
Type: float
elasticsearch.thread_pool.get.rejected

The number of rejected threads in the get pool.
thread / None
Type: float
elasticsearch.thread_pool.get.threads

/
Type: float
elasticsearch.thread_pool.index.active

/
Type: float
elasticsearch.thread_pool.index.queue

/
Type: float
elasticsearch.thread_pool.index.rejected

The number of rejected threads in the index pool.
thread / None
Type: float
elasticsearch.thread_pool.index.threads

/
Type: float
elasticsearch.thread_pool.listener.active

The number of active threads in the listener pool.
thread / None
Type: float
elasticsearch.thread_pool.listener.queue

The number of queued threads in the listener pool.
thread / None
Type: float
elasticsearch.thread_pool.listener.rejected

The number of rejected threads in the listener pool.
thread / None
Type: float
elasticsearch.thread_pool.listener.threads

The total number of threads in the listener pool.
thread / None
Type: float
elasticsearch.thread_pool.management.active

/
Type: float
elasticsearch.thread_pool.management.queue

/
Type: float
elasticsearch.thread_pool.management.rejected

The number of rejected threads in the management pool.
thread / None
Type: float
elasticsearch.thread_pool.management.threads

/
Type: float
elasticsearch.thread_pool.merge.active

The number of active threads in the merge pool.
thread / None
Type: float
elasticsearch.thread_pool.merge.queue

The number of queued threads in the merge pool.
thread / None
Type: float
elasticsearch.thread_pool.merge.rejected

The number of rejected threads in the merge pool.
thread / None
Type: float
elasticsearch.thread_pool.merge.threads

The total number of threads in the merge pool.
thread / None
Type: float
elasticsearch.thread_pool.percolate.active

/
Type: float
elasticsearch.thread_pool.percolate.queue

/
Type: float
elasticsearch.thread_pool.percolate.rejected

The number of rejected threads in the percolate pool.
thread / None
Type: float
elasticsearch.thread_pool.percolate.threads

/
Type: float
elasticsearch.thread_pool.refresh.active

/
Type: float
elasticsearch.thread_pool.refresh.queue

/
Type: float
elasticsearch.thread_pool.refresh.rejected

The number of rejected threads in the refresh pool.
thread / None
Type: float
elasticsearch.thread_pool.refresh.threads

/
Type: float
elasticsearch.thread_pool.search.active

/
Type: float
elasticsearch.thread_pool.search.queue

/
Type: float
elasticsearch.thread_pool.search.rejected

The number of rejected threads in the search pool.
thread / None
Type: float
elasticsearch.thread_pool.search.threads

/
Type: float
elasticsearch.thread_pool.snapshot.active

/
Type: float
elasticsearch.thread_pool.snapshot.queue

/
Type: float
elasticsearch.thread_pool.snapshot.rejected

The number of rejected threads in the snapshot pool.
thread / None
Type: float
elasticsearch.thread_pool.snapshot.threads

/
Type: float
elasticsearch.transport.rx_count

/
Type: float
elasticsearch.transport.rx_size

/
Type: float
elasticsearch.transport.server_open

/
Type: float
elasticsearch.transport.tx_count

/
Type: float
elasticsearch.transport.tx_size

byte /
Type: float
elasticsearch.unassigned_shards

/
Type: float
jvm.gc.collection_count

The total number of garbage collections run by the JVM.
garbage collection / None
Type: float
jvm.gc.collection_time

The total time spent on garbage collection in the JVM.
second / None
Type: float
jvm.gc.collectors.old.collection_time

The total time spent in major GCs in the JVM that collect old generation objects.
second / None
Type: float
jvm.gc.collectors.old.count

The total count of major GCs in the JVM that collect old generation objects.
garbage collection / None
Type: float
jvm.gc.collectors.young.collection_time

The total time spent in minor GCs in the JVM that collects young generation objects.
second / None
Type: float
jvm.gc.collectors.young.count

The total count of minor GCs in the JVM that collects young generation objects.
garbage collection / None
Type: float
jvm.gc.concurrent_mark_sweep.collection_time

The total time spent on "concurrent mark & sweep" GCs in the JVM.
second / None
Type: float
jvm.gc.concurrent_mark_sweep.count

The total count of "concurrent mark & sweep" GCs in the JVM.
garbage collection / None
Type: float
jvm.gc.par_new.collection_time

The total time spent on "parallel new" GCs in the JVM.
second / None
Type: float
jvm.gc.par_new.count

The total count of "parallel new" GCs in the JVM.
garbage collection / None
Type: float
jvm.mem.heap_committed

The amount of memory guaranteed to be available to the JVM heap.
byte / None
Type: float
jvm.mem.heap_in_use

The amount of memory currently used by the JVM heap as a value between 0 and 1.
None / None
Type: float
jvm.mem.heap_max

The maximum amount of memory that can be used by the JVM heap.
byte / None
Type: float
jvm.mem.heap_used

The amount of memory in bytes currently used by the JVM heap.
byte / None
Type: float
jvm.mem.non_heap_committed

The amount of memory guaranteed to be available to JVM non-heap.
byte / None
Type: float
jvm.mem.non_heap_used

The amount of memory in bytes currently used by the JVM non-heap.
byte / None
Type: float
jvm.threads.count

The number of active threads in the JVM.
thread / None
Type: float
jvm.threads.peak_count

The peak number of threads used by the JVM.
thread / None
Type: float
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Monday  —  Friday.

10am  —  6pm UK.

Dedicated Support.