How to collect and extract metrics using Sensu checks

What are Sensu checks?

In short, Sensu checks are commands (or scripts), executed by the Sensu agent, that output data and produce an exit code to indicate a state. If you are unfamiliar with checks, or would like to learn how to configure one first, take a look through the check reference doc and guide before you continue.

Extracting metrics from check output

In order to extract metrics from check output, you’ll need to do the following:

  1. Configure the check command such that the command execution outputs metrics in one of the supported output metric formats.
  2. Configure the check output_metric_format to one of the supported output metric formats.
  3. Configure the check output_metric_handlers (optional) to a Sensu handler that is equipped to handle Sensu metrics (see handlers or influx-db handler to learn more).

You can configure the check with these fields at creation, or use the commands below assuming you have a check named collect-metrics. In this example, we’ll be using graphite_plaintext format and sending the metrics to a handler named influx-db.

sensuctl check set-command collect-metrics
sensuctl check set-output-metric-format collect-metrics graphite_plaintext
sensuctl check set-output-metric-handlers collect-metrics influx-db

Supported output metric formats

The output metric formats that Sensu currently supports for check output metric extraction are nagios, influxdb, graphite, and opentsdb.

output_metric_format nagios_perfdata
documentation Nagios Performance Data
PING ok - Packet loss = 0%, RTA = 0.80 ms | percent_packet_loss=0, rta=0.80
output_metric_format graphite_plaintext
documentation Graphite Plaintext Protocol
local.random.diceroll 4 123456789
output_metric_format influxdb_line
documentation InfluxDB Line Protocol
weather,location=us-midwest temperature=82 1465839830100400200
output_metric_format opentsdb_line
documentation OpenTSDB Data Specification
sys.cpu.user 1356998400 42.5 host=webserver01 cpu=0

Validating the metrics

If the check output is formatted correctly according to its output_metric_format, the metrics will be extracted in Sensu Metric Format, and passed to the event pipeline. You should expect to see logged errors if Sensu is unable to parse the check output. You can validate that metrics have been extracted from your check by inspecting the event passed to the handler. See our troubleshooting guide for an example debug handler which writes events to a file for inspection. The example check we used would yield an event similar to the one below:

type: Event
api_version: core/v2
metadata: {}
      name: collect-metrics
      namespace: default
    output: |-
      cpu.idle_percentage 61 1525462242
      mem.sys 104448 1525462242
    output_metric_format: graphite_plaintext
    - influx-db
    - influx-db
    - name: cpu.idle_percentage
      tags: []
      timestamp: 1525462242
      value: 61
    - name: mem.sys
      tags: []
      timestamp: 1525462242
      value: 104448
  "type": "Event",
  "api_version": "core/v2",
  "metadata": {},
  "spec": {
    "check": {
      "metadata": {
        "name": "collect-metrics",
        "namespace": "default"
      "command": "",
      "output": "cpu.idle_percentage 61 1525462242\nmem.sys 104448 1525462242",
      "output_metric_format": "graphite_plaintext",
      "output_metric_handlers": [
    "metrics": {
      "handlers": [
      "points": [
          "name": "cpu.idle_percentage",
          "value": 61,
          "timestamp": 1525462242,
          "tags": []
          "name": "mem.sys",
          "value": 104448,
          "timestamp": 1525462242,
          "tags": []

Next steps

Now you know how to extract metrics from check output! Check out the below resources for some further reading: