Monitoring MongoDB

This article will help you get the Mongodb plugin for sd-agent configured and returning metrics

Installing the mongodb plugin package

Install the mongodb plugin on Debian/Ubuntu:

sudo apt-get install sd-agent-mongo

Install the mongodb plugin on RHEL/CentOS:

sudo yum install sd-agent-mongo

Read more about agent plugins.

Configuring the agent to monitor MongoDB

1. Connect to the mongo shell and create a monitoring user (note this applies to MongoDB 3.0 upwards):

mongo
use admin
db.createUser({user: "serverdensity", pwd: "supersecretpassword", roles: [ "read","clusterAdmin","readAnyDatabase" ]})

2. Configure /etc/sd-agent/conf.d/mongo.yaml

init_config:
instances:
   # Specify the MongoDB URI, with database to use for reporting (defaults to "admin")
   # E.g. mongodb://localhost:27016/my-db
  - server: mongodb://host:port/db-name
    username: serverdensity
    password: supersecurepassword
  • If you run the database server on a non-standard port, you need to declare the path to the sock or want to execute check, amend the rest of the config file as necessary.
  • NOTE: It is possible to use a connection string in the server option, such as mongodb://serverdensity:supersecurepassword@localhost:27016/my-db. However this is NOT recommended as the server string is automatically used as a tag and this will expose your user and password in the Server Density UI.

3. Restart the agent

sudo /etc/init.d/sd-agent restart

or

sudo systemctl restart sd-agent

Connecting over SSL

We can connect to your MongoDB instances using SSL but it may require some changes to the agent if you are using certificate files.

  • If you are not using certificate files

You can just specify ?ssl=true in the connection string e.g. in step 2 of the installation instructions, then specify mongodb://hostname:port/?ssl=true as part of the config.

  • If you are using certificate files

If you are using cert files then you'll need to tell the agent where they are. Set these 4 lines in your {agentdir}/conf.d/mongo.yaml file:

ssl: True # Optional (default to False)
ssl_keyfile: /path/to/key.file
ssl_certfile: /path/to/cert.file

Verifying the configuration
Execute info to verify the configuration with the following:

sudo /etc/init.d/sd-agent info 

or

/usr/share/python/sd-agent/agent.py info

If the agent has been configured correctly you'll see an output such as:

mongodb
-----
  - instance #0 [OK]
  - Collected * metrics

You can also view the metrics returned with the following command:

sudo -u sd-agent /usr/share/python/sd-agent/agent.py check mongo

Configuring graphs

Click the name of your server from the Devices list in your Server Density account then go to the Metrics tab. Click the + Graph button on the right then choose the mongodb metrics to display the graphs. The metrics will also be available to select when building dashboard graphs.

Screen_Shot_2018-01-18_at_11.46.12.png

Monitored metrics

MetricValues
mongodb.asserts.msgps

/
Type: int
mongodb.asserts.regularps

/
Type: int
mongodb.asserts.rolloversps

/
Type: int
mongodb.asserts.userps

/
Type: int
mongodb.asserts.warningps

/
Type: int
mongodb.backgroundflushing.average_ms

Average time for each flush to disk.
millisecond / None
Type: float
mongodb.backgroundflushing.flushesps

Number of times the database has flushed all writes to disk.
flush / second
Type: float
mongodb.backgroundflushing.last_ms

Amount of time that the last flush operation took to complete.
millisecond / None
Type: float
mongodb.backgroundflushing.total_ms

Total number of time that the `mongod` processes have spent writing (i.e. flushing) data to disk.
millisecond / None
Type: float
mongodb.collection.avgObjSize

The size of the average object in the collection in bytes.
byte / None
Type: float
mongodb.collection.capped

Whether or not the collection is capped.
record / None
Type: float
mongodb.collection.count

Total number of objects in the collection.
item / None
Type: float
mongodb.collection.indexSizes

Size of index in bytes.
byte / None
Type: float
mongodb.collection.max

Maximum number of documents in a capped collection.
document / None
Type: float
mongodb.collection.maxSize

Maximum size of a capped collection in bytes.
byte / None
Type: float
mongodb.collection.nindexes

Total number of indices on the collection.
index / None
Type: float
mongodb.collection.size

The total size in bytes of the data in the collection plus the size of every indexes on the mongodb.collection.
byte / None
Type: float
mongodb.collection.storageSize

Total storage space allocated to this collection for document storage.
byte / None
Type: float
mongodb.connections.available

/
Type: int
mongodb.connections.current

/
Type: int
mongodb.connections.totalcreated

/
Type: int
mongodb.cursors.timedout

/
Type: int
mongodb.cursors.totalopen

/
Type: int
mongodb.dbCount

/
Type: int
mongodb.dbStatsDataSize

/
Type: int
mongodb.dbStatsFileSize

/
Type: int
mongodb.dbStatsIndexSize

/
Type: float
mongodb.dbStatsObjects

/
Type: int
mongodb.dbStatsStorageSize

/
Type: int
mongodb.dur.commits

Number of transactions written to the journal during the last journal group commit interval.
transaction / None
Type: float
mongodb.dur.commitsinwritelock

Count of the commits that occurred while a write lock was held.
commit / None
Type: float
mongodb.dur.compression

Compression ratio of the data written to the journal.
fraction / None
Type: float
mongodb.dur.earlycommits

Number of times MongoDB requested a commit before the scheduled journal group commit interval.
commit / None
Type: float
mongodb.dur.journaledmb

Amount of data written to journal during the last journal group commit interval.
mebibyte / None
Type: float
mongodb.dur.timems.commits

Amount of time spent for commits.
millisecond / None
Type: float
mongodb.dur.timems.commitsinwritelock

Amount of time spent for commits that occurred while a write lock was held.
millisecond / None
Type: float
mongodb.dur.timems.dt

Amount of time over which MongoDB collected the `dur.timeMS` data.
millisecond / None
Type: float
mongodb.dur.timems.preplogbuffer

Amount of time spent preparing to write to the journal.
millisecond / None
Type: float
mongodb.dur.timems.remapprivateview

Amount of time spent remapping copy-on-write memory mapped views.
millisecond / None
Type: float
mongodb.dur.timems.writetodatafiles

Amount of time spent writing to data files after journaling.
millisecond / None
Type: float
mongodb.dur.timems.writetojournal

Amount of time spent writing to the journal
millisecond / None
Type: float
mongodb.dur.writetodatafilesmb

Amount of data written from journal to the data files during the last journal group commit interval.
mebibyte / None
Type: float
mongodb.extra_info.heap_usage_bytesps

/
Type: float
mongodb.extra_info.page_faultsps

/
Type: float
mongodb.globallock.activeclients.readers

Count of the active client connections performing read operations.
connection / None
Type: float
mongodb.globallock.activeclients.total

Total number of active client connections to the database.
connection / None
Type: float
mongodb.globallock.activeclients.writers

Count of active client connections performing write operations.
connection / None
Type: float
mongodb.globallock.currentqueue.readers

Number of operations that are currently queued and waiting for the read lock.
operation / None
Type: float
mongodb.globallock.currentqueue.total

Total number of operations queued waiting for the lock.
operation / None
Type: float
mongodb.globallock.currentqueue.writers

Number of operations that are currently queued and waiting for the write lock.
operation / None
Type: float
mongodb.globallock.locktime

/
Type: int
mongodb.globallock.ratio

/
Type: float
mongodb.globallock.totaltime

/
Type: int
mongodb.indexcounters.accessesps

Number of times that operations have accessed indexes per second.
event / second
Type: float
mongodb.indexcounters.hitsps

Number of times per second that an index has been accessed and mongod is able to return the index from memory.
hit / second
Type: float
mongodb.indexcounters.missesps

Number of times per second that an operation attempted to access an index that was not in memory.
miss / second
Type: float
mongodb.indexcounters.missratio

Ratio of index hits to misses.
fraction / None
Type: float
mongodb.indexcounters.resetsps

Number of times per second the index counters have been reset.
event / second
Type: float
mongodb.locks.collection.acquirecount.exclusiveps

Number of times the collection lock type was aquired in the Exclusive (X) mode.
lock / second
Type: float
mongodb.locks.collection.acquirecount.intent_exclusiveps

Number of times the collection lock type was aquired in the Intent Exclusive (IX) mode.
lock / second
Type: float
mongodb.locks.collection.acquirecount.intent_sharedps

Number of times the collection lock type was aquired in the Intent Shared (IS) mode.
lock / second
Type: float
mongodb.locks.collection.acquirecount.sharedps

Number of times the collection lock type was aquired in the Shared (S) mode.
lock / second
Type: float
mongodb.locks.collection.acquirewaitcount.exclusiveps

Number of times the collection lock type acquisition in the Exclusive (X) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.collection.acquirewaitcount.sharedps

Number of times the collection lock type acquisition in the Shared (S) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.collection.timeacquiringmicros.exclusiveps

Wait time for the collection lock type acquisitions in the Exclusive (X) mode.
fraction / None
Type: float
mongodb.locks.collection.timeacquiringmicros.sharedps

Wait time for the collection lock type acquisitions in the Shared (S) mode.
fraction / None
Type: float
mongodb.locks.database.acquirecount.exclusiveps

Number of times the database lock type was aquired in the Exclusive (X) mode.
lock / second
Type: float
mongodb.locks.database.acquirecount.intent_exclusiveps

Number of times the database lock type was aquired in the Intent Exclusive (IX) mode.
lock / second
Type: float
mongodb.locks.database.acquirecount.intent_sharedps

Number of times the database lock type was aquired in the Intent Shared (IS) mode.
lock / second
Type: float
mongodb.locks.database.acquirecount.sharedps

Number of times the database lock type was aquired in the Shared (S) mode.
lock / second
Type: float
mongodb.locks.database.acquirewaitcount.exclusiveps

Number of times the database lock type acquisition in the Exclusive (X) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.database.acquirewaitcount.intent_exclusiveps

Number of times the database lock type acquisition in the Intent Exclusive (IX) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.database.acquirewaitcount.intent_sharedps

Number of times the database lock type acquisition in the Intent Shared (IS) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.database.acquirewaitcount.sharedps

Number of times the database lock type acquisition in the Shared (S) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.database.timeacquiringmicros.exclusiveps

Wait time for the database lock type acquisitions in the Exclusive (X) mode.
fraction / None
Type: float
mongodb.locks.database.timeacquiringmicros.intent_exclusiveps

Wait time for the database lock type acquisitions in the Intent Exclusive (IX) mode.
fraction / None
Type: float
mongodb.locks.database.timeacquiringmicros.intent_sharedps

Wait time for the database lock type acquisitions in the Intent Shared (IS) mode.
fraction / None
Type: float
mongodb.locks.database.timeacquiringmicros.sharedps

Wait time for the database lock type acquisitions in the Shared (S) mode.
fraction / None
Type: float
mongodb.locks.global.acquirecount.exclusiveps

Number of times the global lock type was aquired in the Exclusive (X) mode.
lock / second
Type: float
mongodb.locks.global.acquirecount.intent_exclusiveps

Number of times the global lock type was aquired in the Intent Exclusive (IX) mode.
lock / second
Type: float
mongodb.locks.global.acquirecount.intent_sharedps

Number of times the global lock type was aquired in the Intent Shared (IS) mode.
lock / second
Type: float
mongodb.locks.global.acquirecount.sharedps

Number of times the global lock type was aquired in the Shared (S) mode.
lock / second
Type: float
mongodb.locks.global.acquirewaitcount.exclusiveps

Number of times the global lock type acquisition in the Exclusive (X) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.global.acquirewaitcount.intent_exclusiveps

Number of times the global lock type acquisition in the Intent Exclusive (IX) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.global.acquirewaitcount.intent_sharedps

Number of times the global lock type acquisition in the Intent Shared (IS) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.global.acquirewaitcount.sharedps

Number of times the global lock type acquisition in the Shared (S) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.global.timeacquiringmicros.exclusiveps

Wait time for the global lock type acquisitions in the Exclusive (X) mode.
fraction / None
Type: float
mongodb.locks.global.timeacquiringmicros.intent_exclusiveps

Wait time for the global lock type acquisitions in the Intent Exclusive (IX) mode.
fraction / None
Type: float
mongodb.locks.global.timeacquiringmicros.intent_sharedps

Wait time for the global lock type acquisitions in the Intent Shared (IS) mode.
fraction / None
Type: float
mongodb.locks.global.timeacquiringmicros.sharedps

Wait time for the global lock type acquisitions in the Shared (S) mode.
fraction / None
Type: float
mongodb.locks.metadata.acquirecount.exclusiveps

Number of times the metadata lock type was aquired in the Exclusive (X) mode.
lock / second
Type: float
mongodb.locks.metadata.acquirecount.sharedps

Number of times the metadata lock type was aquired in the Shared (S) mode.
lock / second
Type: float
mongodb.locks.mmapv1journal.acquirecount.intent_exclusiveps

Number of times the MMAPv1 storage engine lock type was aquired in the Intent Exclusive (IX) mode.
lock / second
Type: float
mongodb.locks.mmapv1journal.acquirecount.intent_sharedps

Number of times the MMAPv1 storage engine lock type was aquired in the Intent Shared (IS) mode.
lock / second
Type: float
mongodb.locks.mmapv1journal.acquirewaitcount.intent_exclusiveps

Number of times the MMAPv1 storage engine lock type acquisition in the Intent Exclusive (IX) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.mmapv1journal.acquirewaitcount.intent_sharedps

Number of times the MMAPv1 storage engine lock type acquisition in the Intent Shared (IS) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.mmapv1journal.timeacquiringmicros.intent_exclusiveps

Wait time for the MMAPv1 storage engine lock type acquisitions in the Intent Exclusive (IX) mode.
fraction / None
Type: float
mongodb.locks.mmapv1journal.timeacquiringmicros.intent_sharedps

Wait time for the MMAPv1 storage engine lock type acquisitions in the Intent Shared (IS) mode.
fraction / None
Type: float
mongodb.locks.oplog.acquirecount.intent_exclusiveps

Number of times the oplog lock type was aquired in the Intent Exclusive (IX) mode.
lock / second
Type: float
mongodb.locks.oplog.acquirecount.sharedps

Number of times the oplog lock type was aquired in the Shared (S) mode.
lock / second
Type: float
mongodb.locks.oplog.acquirewaitcount.intent_exclusiveps

Number of times the oplog lock type acquisition in the Intent Exclusive (IX) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.oplog.acquirewaitcount.sharedps

Number of times the oplog lock type acquisition in the Shared (S) mode encountered waits because the locks were held in a conflicting mode.
wait / second
Type: float
mongodb.locks.oplog.timeacquiringmicros.intent_exclusiveps

Wait time for the oplog lock type acquisitions in the Intent Exclusive (IX) mode.
fraction / None
Type: float
mongodb.locks.oplog.timeacquiringmicros.sharedps

Wait time for the oplog lock type acquisitions in the Shared (S) mode.
fraction / None
Type: float
mongodb.mem.mapped

Amount of mapped memory by the database.
mebibyte / None
Type: float
mongodb.mem.mappedwithjournal

/
Type: int
mongodb.mem.resident

Amount of memory currently used by the database process.
mebibyte / None
Type: float
mongodb.mem.virtual

Amount of virtual memory used by the mongod process.
mebibyte / None
Type: float
mongodb.metrics.cursor.open.notimeout

Number of open cursors with the option `DBQuery.Option.noTimeout` set to prevent timeout after a period of inactivity.
cursor / None
Type: float
mongodb.metrics.cursor.open.pinned

Number of “pinned” open cursors.
cursor / None
Type: float
mongodb.metrics.cursor.open.total

Number of cursors that MongoDB is maintaining for clients.
cursor / None
Type: float
mongodb.metrics.cursor.timedoutps

Number of cursors that time out, per second.
cursor / second
Type: float
mongodb.metrics.document.deletedps

Number of documents deleted per second.
document / second
Type: float
mongodb.metrics.document.insertedps

Number of documents inserted per second.
document / second
Type: float
mongodb.metrics.document.returnedps

Number of documents returned by queries per second.
document / second
Type: float
mongodb.metrics.document.updatedps

Number of documents updated per second.
document / second
Type: float
mongodb.metrics.getlasterror.wtime.numps

Number of getLastError operations per second with a specified write concern (i.e. w) that wait for one or more members of a replica set to acknowledge the write operation.
operation / second
Type: float
mongodb.metrics.getlasterror.wtime.totalmillisps

Fraction of time (ms/s) that the mongod has spent performing getLastError operations with write concern (i.e. w) that wait for one or more members of a replica set to acknowledge the write operation.
fraction / None
Type: float
mongodb.metrics.getlasterror.wtimeoutsps

Number of times per second that write concern operations have timed out as a result of the wtimeout threshold to getLastError
event / second
Type: float
mongodb.metrics.operation.fastmodps

Number of update operations per second that neither cause documents to grow nor require updates to the index.
operation / second
Type: float
mongodb.metrics.operation.idhackps

Number of queries per second that contain the _id field.
query / second
Type: float
mongodb.metrics.operation.scanandorderps

Number of queries per second that return sorted numbers that cannot perform the sort operation using an index.
query / second
Type: float
mongodb.metrics.queryexecutor.scannedps

Number of index items scanned per second during queries and query-plan evaluation.
operation / second
Type: float
mongodb.metrics.record.movesps

Number of times per second documents move within the on-disk representation of the MongoDB data set.
operation / second
Type: float
mongodb.metrics.repl.apply.batches.numps

Number of batches applied across all databases per second.
operation / second
Type: float
mongodb.metrics.repl.apply.batches.totalmillisps

Fraction of time (ms/s) the mongod has spent applying operations from the oplog.
fraction / None
Type: float
mongodb.metrics.repl.apply.opsps

Number of oplog operations applied per second.
operation / second
Type: float
mongodb.metrics.repl.buffer.count

Number of operations in the oplog buffer.
operation / None
Type: float
mongodb.metrics.repl.buffer.maxsizebytes

Maximum size of the buffer.
byte / None
Type: float
mongodb.metrics.repl.buffer.sizebytes

Current size of the contents of the oplog buffer.
byte / None
Type: float
mongodb.metrics.repl.network.bytesps

Amount of data read from the replication sync source per second.
byte / second
Type: float
mongodb.metrics.repl.network.getmores.numps

Number of getmore operations per second.
operation / second
Type: float
mongodb.metrics.repl.network.getmores.totalmillisps

Fraction of time (ms/s) required to collect data from getmore operations.
fraction / None
Type: float
mongodb.metrics.repl.network.opsps

Number of operations read from the replication source per second.
operation / second
Type: float
mongodb.metrics.repl.network.readerscreatedps

Number of oplog query processes created per second.
process / second
Type: float
mongodb.metrics.ttl.deleteddocumentsps

Number of documents deleted from collections with a ttl index per second.
document / second
Type: float
mongodb.metrics.ttl.passesps

Number of times per second the background process removes documents from collections with a ttl index.
operation / second
Type: float
mongodb.opcounters.commandps

/
Type: float
mongodb.opcounters.deleteps

/
Type: float
mongodb.opcounters.getmoreps

/
Type: float
mongodb.opcounters.insertps

/
Type: float
mongodb.opcounters.queryps

/
Type: float
mongodb.opcounters.updateps

/
Type: float
mongodb.opcountersrepl.commandps

/
Type: float
mongodb.opcountersrepl.deleteps

/
Type: float
mongodb.opcountersrepl.getmoreps

/
Type: float
mongodb.opcountersrepl.insertps

/
Type: float
mongodb.opcountersrepl.queryps

/
Type: float
mongodb.opcountersrepl.updateps

/
Type: float
mongodb.oplog.logsizemb

Total size of the oplog.
mebibyte / None
Type: float
mongodb.oplog.timediff

Oplog window: difference between the first and last operation in the oplog.
second / None
Type: float
mongodb.oplog.usedsizemb

Total amount of space used by the oplog.
mebibyte / None
Type: float
mongodb.replset.health

/
Type: int
mongodb.replset.replicationlag

second /
Type: int
mongodb.replset.state

/
Type: int
mongodb.stats.datasize

Total size of the data held in this database including the padding factor.
byte / None
Type: float
mongodb.stats.indexes

Total number of indexes across all collections in the database.
index / None
Type: float
mongodb.stats.indexsize

Total size of all indexes created on this database.
byte / None
Type: float
mongodb.stats.objects

Number of objects (documents) in the database across all collections.
object / None
Type: float
mongodb.stats.storagesize

Total amount of space allocated to collections in this database for document storage.
byte / None
Type: float
mongodb.totalOpenCursors

/
Type: int
mongodb.uptime

second /
Type: int
mongodb.wiredtiger.cache.bytes_currently_in_cache

Size of the data currently in cache.
byte / None
Type: float
mongodb.wiredtiger.cache.failed_eviction_of_pages_exceeding_the_in_memory_maximumps

Number of failed eviction of pages that exceeded the in-memory maximum, per second.
page / second
Type: float
mongodb.wiredtiger.cache.in_memory_page_splits

In-memory page splits.
split / None
Type: float
mongodb.wiredtiger.cache.maximum_bytes_configured

Maximum cache size.
byte / None
Type: float
mongodb.wiredtiger.cache.maximum_page_size_at_eviction

Maximum page size at eviction.
byte / None
Type: float
mongodb.wiredtiger.cache.modified_pages_evicted

Number of pages, that have been modified, evicted from the cache.
page / None
Type: float
mongodb.wiredtiger.cache.pages_currently_held_in_cache

Number of pages currently held in the cache.
page / None
Type: float
mongodb.wiredtiger.cache.pages_evicted_by_application_threadsps

Number of page evicted by application threads per second.
page / second
Type: float
mongodb.wiredtiger.cache.pages_evicted_exceeding_the_in_memory_maximumps

Number of pages evicted because they exceeded the cache in-memory maximum, per second.
page / second
Type: float
mongodb.wiredtiger.cache.tracked_dirty_bytes_in_cache

Size of the dirty data in the cache.
byte / None
Type: float
mongodb.wiredtiger.cache.unmodified_pages_evicted

Number of pages, that were not modified, evicted from the cache.
page / None
Type: float
mongodb.wiredtiger.concurrenttransactions.read.available

Number of available read tickets (concurrent transactions) remaining.
ticket / None
Type: float
mongodb.wiredtiger.concurrenttransactions.read.out

Number of read tickets (concurrent transactions) in use.
ticket / None
Type: float
mongodb.wiredtiger.concurrenttransactions.read.totaltickets

Total number of read tickets (concurrent transactions) available.
ticket / None
Type: float
mongodb.wiredtiger.concurrenttransactions.write.available

Number of available write tickets (concurrent transactions) remaining.
ticket / None
Type: float
mongodb.wiredtiger.concurrenttransactions.write.out

Number of write tickets (concurrent transactions) in use.
ticket / None
Type: float
mongodb.wiredtiger.concurrenttransactions.write.totaltickets

Total number of write tickets (concurrent transactions) available.
ticket / None
Type: float
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Monday  —  Friday.

10am  —  6pm UK.

Dedicated Support.