Sensu agent

Example Sensu agent configuration file (download)

The Sensu agent is a lightweight client that runs on the infrastructure components you want to monitor. Agents register with the Sensu backend as monitoring entities with type: "agent". Agent entities are responsible for creating check and metrics events to send to the backend event pipeline. The Sensu agent is available for Linux, macOS, and Windows. See the installation guide to install the agent.

Communication between the agent and backend

The Sensu agent uses WebSocket (ws) protocol to send and receive JSON messages with the Sensu backend. For optimal network throughput, agents will attempt to negotiate the use of Protobuf serialization when communicating with a Sensu backend that supports it. This communication is via clear text by default. Follow Secure Sensu to configure the backend and agent for WebSocket Secure (wss) encrypted communication.

Create monitoring events using service checks

Sensu uses the publish/subscribe pattern of communication, which allows automated registration and deregistration of ephemeral systems. At the core of this model are Sensu agent subscriptions.

Each Sensu agent has a defined set of subscriptions: a list of roles and responsibilities assigned to the system (for example, a webserver or database). These subscriptions determine which monitoring checks the agent will execute. Agent subscriptions allow Sensu to request check executions on a group of systems at a time instead of a traditional 1:1 mapping of configured hosts to monitoring checks. For an agent to execute a service check, you must specify the same subscription in the agent configuration and the check definition.

After receiving a check request from the Sensu backend, the agent:

  1. Applies any tokens that match attribute values in the check definition.
  2. Fetches assets and stores them in its local cache. By default, agents cache asset data at /var/cache/sensu/sensu-agent (C:\ProgramData\sensu\cache\sensu-agent on Windows systems) or as specified by the the cache-dir flag.
  3. Executes the check command.
  4. Executes any hooks specified by the check based on the exit status.
  5. Creates an event that contains information about the applicable entity, check, and metric.

Subscription configuration

To configure subscriptions for an agent, set the subscriptions flag. To configure subscriptions for a check, set the check definition attribute subscriptions.

In addition to the subscriptions defined in the agent configuration, Sensu agent entities also subscribe automatically to subscriptions that match their entity name. For example, an agent entity with name: "i-424242" subscribes to check requests with the subscription entity:i-424242. This makes it possible to generate ad hoc check requests that target specific entities via the API.

Proxy entities

Sensu proxy entities allow Sensu to monitor external resources on systems or devices where a Sensu agent cannot be installed (such a network switch). The Sensu backend stores proxy entity definitions (unlike agent entities, which the agent stores). When the backend requests a check that includes a proxy_entity_name, the agent includes the provided entity information in the event data in place of the agent entity data. See the entity reference and Monitor external resources for more information about monitoring proxy entities.

Create monitoring events using the agent API

The Sensu agent API allows external sources to send monitoring data to Sensu without requiring the external sources to know anything about Sensu’s internal implementation. The agent API listens on the address and port specified by the API configuration flags. Only unsecured HTTP (no HTTPS) is supported at this time. Any requests for unknown endpoints result in an HTTP 404 Not Found response.

/events (POST)

The /events API provides HTTP POST access to publish monitoring events to the Sensu backend pipeline via the agent API. The agent places events created via the /events POST endpoint into a queue stored on disk. In case of a loss of connection with the backend or agent shutdown, the agent preserves queued event data. When the connection is reestablished, the agent sends the queued events to the backend.

The /events API uses a configurable burst limit and rate limit for relaying events to the backend. See API configuration flags to configure the events-burst-limit and events-rate-limit flags.

Example POST request to events API

The following example submits an HTTP POST request to the /events API. The request creates event for a check named check-mysql-status with the output could not connect to mysql and a status of 1 (warning). The agent responds with an HTTP 202 Accepted response to indicate that the event has been added to the queue to be sent to the backend.

The event will be handled according to an email handler definition.

NOTE: For HTTP POST requests to the agent /events API, check spec attributes are not required. When doing so, the spec attributes (including handlers) are listed as individual top-level attributes in the check definition instead.

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "handlers": ["email"],
    "status": 1,
    "output": "could not connect to mysql"
  }
}' \
http://127.0.0.1:3031/events

HTTP/1.1 202 Accepted

PRO TIP: To use the agent API /events endpoint to create proxy entities, include a proxy_entity_name attribute within the check scope.

Detect silent failures

You can use the Sensu agent API in combination with the check time-to-live (TTL) attribute to detect silent failures. This creates what’s commonly referred to as a “dead man’s switch”.

With check TTLs, Sensu can set an expectation that a Sensu agent will publish additional events for a check within the period of time specified by the TTL attribute. If a Sensu agent fails to publish an event before the check TTL expires, the Sensu backend creates an event with a status of 1 (warning) to indicate the expected event was not received. For more information about check TTLs, see the the check reference.

You can use the Sensu agent API to enable tasks that run outside of Sensu’s check scheduling to emit events. Using the check TTL attribute, these events create a dead man’s switch: if the task fails for any reason, the lack of an “all clear” event from the task will notify operators of a silent failure (which might otherwise be missed). If an external source sends a Sensu event with a check TTL to the Sensu agent API, Sensu expects another event from the same external source before the TTL expires.

In this example, external event input via the Sensu agent API uses a check TTL to create a dead man’s switch for MySQL backups. Assume that a MySQL backup script runs periodically, and you expect the job to take a little less than 7 hours to complete.

  • If the job completes successfully, you want a record of it, but you don’t need to receive an alert.
  • If the job fails or continues running longer than the expected 7 hours, you do need to receive an alert.

This script sends an event that tells the Sensu backend to expect an additional event with the same name within 7 hours of the first event:

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "mysql-backup-job"
    },
    "status": 0,
    "output": "mysql backup initiated",
    "ttl": 25200
  }
}' \
http://127.0.0.1:3031/events

With this initial event submitted to the agent API, you recorded in the Sensu backend that your script started. You also configured the dead man’s switch so that you’ll receive an alert if the job fails or runs for too long. Although it is possible for your script to handle errors gracefully and emit additional monitoring events, this approach allows you to worry less about handling every possible error case. A lack of additional events before the 7-hour period elapses results in an alert.

If your backup script runs successfully, you can send an additional event without the TTL attribute, which removes the dead man’s switch:

curl -X POST \
-H 'Content-Type: application/json' \
-d '{
  "check": {
    "metadata": {
      "name": "mysql-backup-job"
    },
    "status": 0,
    "output": "mysql backup ran successfully!"
  }
}' \
http://127.0.0.1:3031/events

When you omit the TTL attribute from this event, you also remove the dead man’s switch being monitored by the Sensu backend. This effectively sounds the “all clear” for this iteration of the task.

API specification

/events (POST)
description Accepts JSON event data and passes the event to the Sensu backend event pipeline for processing.
example url http://hostname:3031/events
payload example
{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "status": 1,
    "output": "could not connect to mysql"
  }
}
payload attributes
    Required:
  • check: All check data must be within the check scope
  • metadata: The check scope must contain a metadata scope
  • name: The metadata scope must contain the name attribute with a string that represents the name of the monitoring check
response codes
  • Success: 202 (Accepted)
  • Malformed: 400 (Bad Request)
  • Error: 500 (Internal Server Error)

/healthz (GET)

The /healthz API provides HTTP GET access to the status of the Sensu agent via the agent API.

Example

In the following example, an HTTP GET request is submitted to the /healthz API:

curl http://127.0.0.1:3031/healthz

The request results in a healthy response:

ok

API specification

/healthz (GET)
description Returns the agent status: ok if the agent is active and connected to a Sensu backend or sensu backend unavailable if the agent cannot connect to a backend.
example url http://hostname:3031/healthz

Create monitoring events using the StatsD listener

Sensu agents include a listener to send StatsD metrics to the event pipeline. By default, Sensu agents listen on UDP socket 8125 for messages that follow the StatsD line protocol and send metric events for handling by the Sensu backend.

For example, you can use the Netcat utility to send metrics to the StatsD listener:

echo 'abc.def.g:10|c' | nc -w1 -u localhost 8125

Sensu does not store metrics received through the StatsD listener, so it’s important to configure event handlers.

StatsD line protocol

The Sensu StatsD listener accepts messages formatted according to the StatsD line protocol:

<metricname>:<value>|<type>

For more information, see the StatsD documentation.

Configure the StatsD listener

To configure the StatsD listener, specify the statsd-event-handlers configuration flag in the agent configuration, and start the agent.

# Start an agent that sends StatsD metrics to InfluxDB
sensu-agent --statsd-event-handlers influx-db

Use the StatsD configuration flags to change the default settings for the StatsD listener address, port, and flush interval.

# Start an agent with a customized address and flush interval
sensu-agent --statsd-event-handlers influx-db --statsd-flush-interval 1 --statsd-metrics-host 123.4.5.11 --statsd-metrics-port 8125

Create monitoring events using the agent TCP and UDP sockets

NOTE: The agent TCP and UDP sockets are deprecated in favor of the agent API.

Sensu agents listen for external monitoring data using TCP and UDP sockets. The agent sockets accept JSON event data and pass events to the Sensu backend event pipeline for processing. The TCP and UDP sockets listen on the address and port specified by the socket configuration flags.

Use the TCP socket

This example demonstrates external monitoring data input via the Sensu agent TCP socket. The example uses Bash’s built-in /dev/tcp file to communicate with the Sensu agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' > /dev/tcp/localhost/3030

You can also use the Netcat utility to send monitoring data to the agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' | nc localhost 3030

Use the UDP socket

This example demonstrates external monitoring data input via the Sensu agent UDP socket. The example uses Bash’s built-in /dev/udp file to communicate with the Sensu agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' > /dev/udp/127.0.0.1/3030

You can also use the Netcat utility to send monitoring data to the agent socket:

echo '{"name": "check-mysql-status", "status": 1, "output": "error!"}' | nc -u -v 127.0.0.1 3030

Socket event format

The agent TCP and UDP sockets use a special event data format designed for backward compatibility with Sensu Core 1.x check results. Attributes specified in socket events appear in the resulting event data passed to the Sensu backend.

Example socket input: Minimum required attributes

{
  "name": "check-mysql-status",
  "status": 1,
  "output": "error!"
}

Example socket input: All attributes

{
  "name": "check-http",
  "status": 1,
  "output": "404",
  "source": "sensu-docs-site",
  "executed": 1550013435,
  "duration": 1.903135228,
  "handlers": ["slack", "influxdb"]
}

Socket event specification

NOTE: The Sensu agent socket ignores any attributes that are not included in this specification.

name
description Check name.
required true
type String
example
"name": "check-mysql-status"
status
description Check execution exit status code. An exit status code of 0 (zero) indicates OK, 1 indicates WARNING, and 2 indicates CRITICAL. Exit status codes other than 0, 1, and 2 indicate an UNKNOWN or custom status.
required true
type Integer
example
"status": 0
output
description Output produced by the check command.
required true
type String
example
"output": "CheckHttp OK: 200, 78572 bytes"
source
description Name of the Sensu entity associated with the event. Use this attribute to tie the event to a proxy entity. If no matching entity exists, Sensu creates a proxy entity with the name provided by the source attribute.
required false
default The agent entity that receives the event data.
type String
example
"source": "sensu-docs-site"
client
description Name of the Sensu entity associated with the event. Use this attribute to tie the event to a proxy entity. If no matching entity exists, Sensu creates a proxy entity with the name provided by the client attribute.

NOTE: The client attribute is deprecated in favor of the source attribute (see above).

required false
default The agent entity that receives the event data.
type String
example
"client": "sensu-docs-site"
executed
description Time at which the check was executed. In seconds since the Unix epoch.
required false
default The time the event was received by the agent.
type Integer
example
"executed": 1458934742
duration
description Amount of time it took to execute the check. In seconds.
required false
type Float
example
"duration": 1.903135228
command
description Command executed to produce the event. Use the command attribute to add context to the event data. Sensu does not execute the command included in this attribute.
required false
type String
example
"command": "check-http.rb -u https://sensuapp.org"
interval
description Interval used to produce the event. Use the interval attribute to add context to the event data. Sensu does not act on the value provided in this attribute.
required false
default 1
type Integer
example
"interval": 60
handlers
description Array of Sensu handler names to use for handling the event. Each handler name in the array must be a string.
required false
type Array
example
"handlers": ["slack", "influxdb"]

Keepalive monitoring

Sensu keepalives are the heartbeat mechanism used to ensure that all registered agents are operational and able to reach the Sensu backend. Sensu agents publish keepalive events containing entity configuration data to the Sensu backend according to the interval specified by the keepalive-interval flag.

If a Sensu agent fails to send keepalive events over the period specified by the keepalive-critical-timeout flag, the Sensu backend creates a keepalive critical alert in the Sensu web UI. The keepalive-critical-timeout is set to 0 (disabled) by default to help ensure that it will not interfere with your keepalive-warning-timeout setting.

If a Sensu agent fails to send keepalive events over the period specified by the keepalive-warning-timeout flag, the Sensu backend creates a keepalive warning alert in the Sensu web UI. The value you specify for keepalive-warning-timeout must be lower than the value you specify for keepalive-critical-timeout.

You can use keepalives to identify unhealthy systems and network partitions, send notifications, and trigger auto-remediation, among other useful actions.

NOTE: Automatic keepalive monitoring is not supported for proxy entities because they cannot run a Sensu agent. You can use the events API to send manual keepalive events for proxy entities.

Handle keepalive events

You can use a keepalive handler to connect keepalive events to your monitoring workflows. Sensu looks for an event handler named keepalive and automatically uses it to process keepalive events.

Suppose you want to receive Slack notifications for keepalive alerts, and you already have a Slack handler set up to process events. To process keepalive events using the Slack pipeline, create a handler set named keepalive and add the slack handler to the handlers array. The resulting keepalive handler set configuration looks like this:

type: Handler
api_version: core/v2
metadata:
  name: keepalive
  namespace: default
spec:
  handlers:
  - slack
  type: set
{
  "type": "Handler",
  "api_version": "core/v2",
  "metadata" : {
    "name": "keepalive",
    "namespace": "default"
  },
  "spec": {
    "type": "set",
    "handlers": [
      "slack"
    ]
  }
}

You can also use the keepalive-handlers flag to send keepalive events to any handler you have configured. If you do not specify a keepalive handler with the keepalive-handlers flag, the Sensu backend will use the default keepalive handler and create an event in sensuctl and the Sensu web UI.

Service management

Start the service

Use the sensu-agent tool to start the agent and apply configuration flags.

Linux

To start the agent with configuration flags:

sensu-agent start --subscriptions disk-checks --log-level debug

To see available configuration flags and defaults:

sensu-agent start --help

To start the agent using a service manager:

sudo service sensu-agent start

If you do not provide any configuration flags, the agent loads configuration from the location specified by the config-file attribute (default is /etc/sensu/agent.yml).

Windows

Run the following command as an admin to install and start the agent:

sensu-agent service install

By default, the agent loads configuration from %ALLUSERSPROFILE%\sensu\config\agent.yml (for example, C:\ProgramData\sensu\config\agent.yml) and stores service logs to %ALLUSERSPROFILE%\sensu\log\sensu-agent.log (for example, C:\ProgramData\sensu\log\sensu-agent.log).

Configure the configuration file and log file locations using the config-file and log-file flags:

sensu-agent service install --config-file 'C:\\monitoring\\sensu\\config\\agent.yml' --log-file 'C:\\monitoring\\sensu\\log\\sensu-agent.log'

Stop the service

To stop the agent service using a service manager:

Linux

sudo service sensu-agent stop

Windows

sc.exe stop SensuAgent

Restart the service

You must restart the agent to implement any configuration updates.

To restart the agent using a service manager:

Linux

sudo service sensu-agent restart

Windows

sc.exe stop SensuAgent
sc.exe start SensuAgent

Enable on boot

To enable the agent to start on system boot:

Linux

sudo systemctl enable sensu-agent

To disable the agent from starting on system boot:

sudo systemctl disable sensu-agent

NOTE: On older distributions of Linux, use sudo chkconfig sensu-agent on to enable the agent and sudo chkconfig sensu-agent off to disable the agent.

Windows

The service is configured to start automatically on boot by default.

Get service status

To see the status of the agent service using a service manager:

Linux

service sensu-agent status

Windows

sc.exe query SensuAgent

Get service version

There are two ways to get the current agent version: the sensu-agent tool and the agent version API.

To get the version of the current sensu-agent tool:

sensu-agent version

To get the version of the running sensu-agent service:

curl http://127.0.0.1:3031/version

Uninstall the service

Windows

sensu-agent service uninstall

Get help

The sensu-agent tool provides general and command-specific help flags:

# Show sensu-agent commands
sensu-agent help

# Show options for the sensu-agent start subcommand
sensu-agent start --help

Registration

In practice, agent registration happens when a Sensu backend processes an agent keepalive event for an agent that is not already registered in the Sensu agent registry (based on the configured agent name). The Sensu backend stores this agent registry, and it is accessible via sensuctl entity list.

All Sensu agent data provided in keepalive events gets stored in the agent registry and used to add context to Sensu events and detect Sensu agents in an unhealthy state.

Registration events

If a Sensu event handler named registration is configured, the Sensu backend creates and processes an event for agent registration, applying any configured filters and mutators before executing the configured handler.

PRO TIP: Use a handler set to execute multiple handlers in response to registration events.

You can use registration events to execute one-time handlers for new Sensu agents. For example, you can use registration event handlers to update external configuration management databases (CMDBs) such as ServiceNow.

The handlers reference includes an example registration event handler.

WARNING: Registration events are not stored in the event registry, so they are not accessible via the Sensu API. However, all registration events are logged in the Sensu backend log.

Deregistration events

As with registration events, the Sensu backend can create and process a deregistration event when the Sensu agent process stops. You can use deregistration events to trigger a handler that updates external CMDBs or performs an action to update ephemeral infrastructures. To enable deregistration events, use the deregister flag, and specify the event handler using the deregistration-handler flag. You can specify a deregistration handler per agent using the deregistration-handler agent flag or by setting a default for all agents using the deregistration-handler backend configuration flag.

Cluster

Agents can connect to a Sensu cluster by specifying any Sensu backend URL in the cluster in the backend-url configuration flag. For more information about clustering, see Backend datastore configuration flags and Run a Sensu cluster.

Synchronize time

System clocks between agents and the backend should be synchronized to a central NTP server. If system time is out-of-sync, it may cause issues with keepalive, metric, and check alerts.

Configuration via flags

The agent loads configuration upon startup, so you must restart the agent for any configuration updates to take effect.

Linux

Specify the agent configuration with either a .yml file or sensu-agent start command line flags. Configuration via command line flags overrides attributes specified in a configuration file. See the Example Sensu agent configuration file for flags and defaults.

Certificate bundles or chains

The Sensu agent supports all types of certificate bundles (or chains) as long as the agent (or leaf) certificate is the first certificate in the bundle. This is because the Go standard library assumes that the first certificate listed in the PEM file is the leaf certificate — the certificate that the program will use to show its own identity.

If you send the leaf certificate alone instead of sending the whole bundle with the leaf certificate first, you will see a certificate not signed by trusted authority error. You must present the whole chain to the remote so it can determine whether it trusts the presented certificate through the chain.

Configuration summary

IMPORTANT: Process discovery is disabled in release 5.20.2. As of 5.20.2, the --discover-processes flag is not available, and new events will not include data in the processes attributes. Instead, the field will be empty: "processes": null.

$ sensu-agent start --help
start the sensu agent

Usage:
  sensu-agent start [flags]

Flags:
      --allow-list string                     path to agent execution allow list configuration file
      --annotations stringToString            entity annotations map (default [])
      --api-host string                       address to bind the Sensu client HTTP API to (default "127.0.0.1")
      --api-port int                          port the Sensu client HTTP API listens on (default 3031)
      --assets-burst-limit int                asset fetch burst limit (default 100)
      --assets-rate-limit float               maximum number of assets fetched per second
      --backend-url strings                   ws/wss URL of Sensu backend server (to specify multiple backends use this flag multiple times) (default [ws://127.0.0.1:8081])
      --cache-dir string                      path to store cached data (default "/var/cache/sensu/sensu-agent")
      --cert-file string                      TLS certificate in PEM format
  -c, --config-file string                    path to sensu-agent config file
      --deregister                            ephemeral agent
      --deregistration-handler string         deregistration handler that should process the entity deregistration event.
      --detect-cloud-provider                 enable cloud provider detection mechanisms
      --disable-assets                        disable check assets on this agent
      --disable-api                           disable the Agent HTTP API
      --disable-sockets                       disable the Agent TCP and UDP event sockets
      --discover-processes                    indicates whether process discovery should be enabled
      --events-burst-limit                    /events api burst limit
      --events-rate-limit                     maximum number of events transmitted to the backend through the /events api
  -h, --help                                  help for start
      --insecure-skip-tls-verify              skip ssl verification
      --keepalive-critical-timeout uint32     number of seconds until agent is considered dead by backend to create a critical event (default 0)
      --keepalive-handlers string             comma-delimited list of keepalive handlers for this entity. This flag can also be invoked multiple times
      --keepalive-interval uint32             number of seconds to send between keepalive events (default 20)
      --keepalive-warning-timeout uint32      number of seconds until agent is considered dead by backend to create a warning event (default 120)
      --key-file string                       TLS certificate key in PEM format
      --labels stringToString                 entity labels map (default [])
      --log-level string                      logging level [panic, fatal, error, warn, info, debug] (default "info")
      --name string                           agent name (defaults to hostname) (default "my-hostname")
      --namespace string                      agent namespace (default "default")
      --password string                       agent password (default "P@ssw0rd!")
      --redact string                         comma-delimited customized list of fields to redact
      --require-fips                          indicates whether fips support should be required in openssl
      --require-openssl                       indicates whether openssl should be required instead of go's built-in crypto
      --socket-host string                    address to bind the Sensu client socket to (default "127.0.0.1")
      --socket-port int                       port the Sensu client socket listens on (default 3030)
      --statsd-disable                        disables the statsd listener and metrics server
      --statsd-event-handlers strings         comma-delimited list of event handlers for statsd metrics
      --statsd-flush-interval int             number of seconds between statsd flush (default 10)
      --statsd-metrics-host string            address used for the statsd metrics server (default "127.0.0.1")
      --statsd-metrics-port int               port used for the statsd metrics server (default 8125)
      --subscriptions string                  comma-delimited list of agent subscriptions
      --trusted-ca-file string                tls certificate authority
      --user string                           agent user (default "agent")

Windows

You can specify the agent configuration using a .yml file. See the example agent configuration file (also provided with Sensu packages at %ALLUSERSPROFILE%\sensu\config\agent.yml.example; default C:\ProgramData\sensu\config\agent.yml.example).

General configuration flags

allow-list
description Path to yaml or json file that contains the allow list of check or hook commands the agent can execute. See allow list configuration commands and the example allow list configuration file for information about building a configuration file.
type String
default ""
environment variable SENSU_ALLOW_LIST
example
# Command line example
sensu-agent start --allow-list /etc/sensu/check-allow-list.yaml

# /etc/sensu/agent.yml example
allow-list: /etc/sensu/check-allow-list.yaml
annotations
description Non-identifying metadata to include with event data that you can access with event filters and tokens. You can use annotations to add data that is meaningful to people or external tools that interact with Sensu.

In contrast to labels, you cannot use annotations in API response filtering, sensuctl response filtering, or web UI view filtering.
required false
type Map of key-value pairs. Keys and values can be any valid UTF-8 string.
default null
environment variable SENSU_ANNOTATIONS
example
# Command line examples
sensu-agent start --annotations sensu.io/plugins/slack/config/webhook-url=https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX
sensu-agent start --annotations example-key="example value" --annotations example-key2="example value"

# /etc/sensu/agent.yml example
annotations:
  sensu.io/plugins/slack/config/webhook-url: "https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX"
assets-burst-limit
description Maximum amount of burst allowed in a rate interval when fetching assets.
type Integer
default 100
environment variable SENSU_ASSETS_BURST_LIMIT
example
# Command line example
sensu-agent start --assets-burst-limit 100

# /etc/sensu/agent.yml example
assets-burst-limit: 100
assets-rate-limit
description Maximum number of assets to fetch per second. The default value 1.39 is equivalent to approximately 5000 user-to-server requests per hour.
type Float
default 1.39
environment variable SENSU_ASSETS_RATE_LIMIT
example
# Command line example
sensu-agent start --assets-rate-limit 1.39

# /etc/sensu/agent.yml example
assets-rate-limit: 1.39
backend-url
description ws or wss URL of the Sensu backend server. To specify multiple backends with sensu-agent start, use this flag multiple times.

NOTE: If you do not specify a port for your backend-url values, the agent will automatically append the default backend port (8081).

type List
default ws://127.0.0.1:8081
environment variable SENSU_BACKEND_URL
example
# Command line examples
sensu-agent start --backend-url ws://0.0.0.0:8081
sensu-agent start --backend-url ws://0.0.0.0:8081 --backend-url ws://0.0.0.0:8082

# /etc/sensu/agent.yml example
backend-url:
  - "ws://0.0.0.0:8081"
  - "ws://0.0.0.0:8082"
  

cache-dir
description Path to store cached data.
type String
default
  • Linux: /var/cache/sensu/sensu-agent
  • Windows: C:\ProgramData\sensu\cache\sensu-agent
environment variable SENSU_CACHE_DIR
example
# Command line example
sensu-agent start --cache-dir /cache/sensu-agent

# /etc/sensu/agent.yml example
cache-dir: "/cache/sensu-agent"
config-file
description Path to Sensu agent configuration file.
type String
default
  • Linux: /etc/sensu/agent.yml
  • FreeBSD: /usr/local/etc/sensu/agent.yml
  • Windows: C:\ProgramData\sensu\config\agent.yml
environment variable SENSU_CONFIG_FILE
example
# Command line example
sensu-agent start --config-file /sensu/agent.yml
sensu-agent start -c /sensu/agent.yml

# /etc/sensu/agent.yml example
config-file: "/sensu/agent.yml"

disable-assets
description When set to true, disables assets for the agent. If an agent attempts to execute a check that requires an asset, the agent will respond with a status of 3 and a message that indicates the agent could not execute the check because assets are disabled.
type Boolean
default false
environment variable SENSU_DISABLE_ASSETS
example
# Command line example
sensu-agent start --disable-assets

# /etc/sensu/agent.yml example
disable-assets: true

discover-processes
description When set to true, the agent populates the processes field in entity.system and updates every 20 seconds.

COMMERCIAL FEATURE: Access the discover-processes flag in the packaged Sensu Go distribution. For more information, see Get started with commercial features.

IMPORTANT: Process discovery is disabled in release 5.20.2. As of 5.20.2, the --discover-processes flag is not available, and new events will not include data in the processes attributes. Instead, the field will be empty: "processes": null.

type Boolean
default false
environment variable SENSU_DISCOVER_PROCESSES
example
# Command line example
sensu-agent start --discover-processes

# /etc/sensu/agent.yml example
discover-processes: true
labels
description Custom attributes to include with event data that you can use for response and web UI view filtering.

If you include labels in your event data, you can filter API responses, sensuctl responses, and web UI views based on them. In other words, labels allow you to create meaningful groupings for your data.

Limit labels to metadata you need to use for response filtering. For complex, non-identifying metadata that you will not need to use in response filtering, use annotations rather than labels.
required false
type Map of key-value pairs. Keys can contain only letters, numbers, and underscores and must start with a letter. Values can be any valid UTF-8 string.
default null
environment variable SENSU_LABELS
example
# Command line examples
sensu-agent start --labels proxy_type=website
sensu-agent start --labels example_key1="example value" example_key2="example value"

# /etc/sensu/agent.yml example
labels:
  proxy_type: "website"

name
description Entity name assigned to the agent entity.
type String
default Defaults to hostname (for example, sensu-centos).
environment variable SENSU_NAME
example
# Command line example
sensu-agent start --name agent-01

# /etc/sensu/agent.yml example
name: "agent-01" 

log-level
description Logging level: panic, fatal, error, warn, info, or debug.
type String
default info
environment variable SENSU_LOG_LEVEL
example
# Command line example
sensu-agent start --log-level debug

# /etc/sensu/agent.yml example
log-level: "debug"

subscriptions
description Array of agent subscriptions that determine which monitoring checks the agent will execute. The subscriptions array items must be strings.
type List
environment variable SENSU_SUBSCRIPTIONS
example
# Command line examples
sensu-agent start --subscriptions disk-checks,process-checks
sensu-agent start --subscriptions disk-checks --subscriptions process-checks

# /etc/sensu/agent.yml example
subscriptions:
  - disk-checks
  - process-checks

API configuration flags

api-host
description Bind address for the Sensu agent HTTP API.
type String
default 127.0.0.1
environment variable SENSU_API_HOST
example
# Command line example
sensu-agent start --api-host 0.0.0.0

# /etc/sensu/agent.yml example
api-host: "0.0.0.0"
api-port
description Listening port for the Sensu agent HTTP API.
type Integer
default 3031
environment variable SENSU_API_PORT
example
# Command line example
sensu-agent start --api-port 4041

# /etc/sensu/agent.yml example
api-port: 4041
disable-api
description true to disable the agent HTTP API. Otherwise, false.
type Boolean
default false
environment variable SENSU_DISABLE_API
example
# Command line example
sensu-agent start --disable-api

# /etc/sensu/agent.yml example
disable-api: true
events-burst-limit
description Maximum amount of burst allowed in a rate interval for the agent events API.
type Integer
default 10
environment variable SENSU_EVENTS_BURST_LIMIT
example
# Command line example
sensu-agent start --events-burst-limit 20

# /etc/sensu/agent.yml example
events-burst-limit: 20
events-rate-limit
description Maximum number of events per second that can be transmitted to the backend with the agent events API.
type Float
default 10.0
environment variable SENSU_EVENTS_RATE_LIMIT
example
# Command line example
sensu-agent start --events-rate-limit 20.0

# /etc/sensu/agent.yml example
events-rate-limit: 20.0

Ephemeral agent configuration flags

deregister
description true if a deregistration event should be created upon Sensu agent process stop. Otherwise, false.
type Boolean
default false
environment variable SENSU_DEREGISTER
example
# Command line example
sensu-agent start --deregister

# /etc/sensu/agent.yml example
deregister: true
deregistration-handler
description Name of a deregistration handler that processes agent deregistration events. This flag overrides any handlers applied by the deregistration-handler backend configuration flag.
type String
environment variable SENSU_DEREGISTRATION_HANDLER
example
# Command line example
sensu-agent start --deregistration-handler deregister

# /etc/sensu/agent.yml example
deregistration-handler: "deregister"

detect-cloud-provider
description true to enable cloud provider detection mechanisms. Otherwise, false. When this flag is enabled, the agent will attempt to read files, resolve hostnames, and make HTTP requests to determine what cloud environment it is running in.
type Boolean
default false
environment variable SENSU_DETECT_CLOUD_PROVIDER
example
# Command line example
sensu-agent start --detect-cloud-provider false

# /etc/sensu/agent.yml example
detect-cloud-provider: "false"

Keepalive configuration flags

keepalive-critical-timeout
description Number of seconds after a missing keepalive event until the agent is considered unresponsive by the Sensu backend to create a critical event. Set to disabled (0) by default. If the value is not 0, it must be greater than or equal to 5.
type Integer
default 0
environment variable SENSU_KEEPALIVE_CRITICAL_TIMEOUT
example
# Command line example
sensu-agent start --keepalive-critical-timeout 300

# /etc/sensu/agent.yml example
keepalive-critical-timeout: 300

keepalive-handlers
description Keepalive event handlers to use for the entity, specified in a comma-delimited list. You can specify any configured handler and invoke the keepalive-handlers flag multiple times. If keepalive handlers are not specified, the Sensu backend will use the default keepalive handler and create an event in sensuctl and the Sensu web UI.
type List
default keepalive
environment variable SENSU_KEEPALIVE_HANDLERS
example
# Command line example
sensu-agent start --keepalive-handlers slack,email

# /etc/sensu/agent.yml example
keepalive-handlers: slack,email
keepalive-interval
description Number of seconds between keepalive events.
type Integer
default 20
environment variable SENSU_KEEPALIVE_INTERNAL
example
# Command line example
sensu-agent start --keepalive-interval 30

# /etc/sensu/agent.yml example
keepalive-interval: 30
keepalive-warning-timeout
description Number of seconds after a missing keepalive event until the agent is considered unresponsive by the Sensu backend to create a warning event. Value must be lower than the keepalive-critical-timeout value. Minimum value is 5.
type Integer
default 120
environment variable SENSU_KEEPALIVE_WARNING_TIMEOUT
example
# Command line example
sensu-agent start --keepalive-warning-timeout 300

# /etc/sensu/agent.yml example
keepalive-warning-timeout: 300

Security configuration flags

namespace
description Agent namespace.

NOTE: Agents are represented in the backend as a class of entity. Entities can only belong to a single namespace.

type String
default default
environment variable SENSU_NAMESPACE
example
# Command line example
sensu-agent start --namespace ops

# /etc/sensu/agent.yml example
namespace: "ops"
user
description Sensu RBAC username used by the agent. Agents require get, list, create, update, and delete permissions for events across all namespaces.
type String
default agent
environment variable SENSU_USER
example
# Command line example
sensu-agent start --user agent-01

# /etc/sensu/agent.yml example
user: "agent-01"
password
description Sensu RBAC password used by the agent.
type String
default P@ssw0rd!
environment variable SENSU_PASSWORD
example
# Command line example
sensu-agent start --password secure-password

# /etc/sensu/agent.yml example
password: "secure-password"
redact
description List of fields to redact when displaying the entity.

NOTE: Redacted secrets are sent via the WebSocket connection and stored in etcd. They are not logged or displayed via the Sensu API.

type List
default By default, Sensu redacts the following fields: password, passwd, pass, api_key, api_token, access_key, secret_key, private_key, secret.
environment variable SENSU_REDACT
example
# Command line example
sensu-agent start --redact secret,ec2_access_key

# /etc/sensu/agent.yml example
redact:
  - secret
  - ec2_access_key
cert-file
description Path to the agent certificate file used in mutual TLS authentication. Sensu supports certificate bundles (or chains) as long as the agent (or leaf) certificate is the first certificate in the bundle.
type String
default ""
environment variable SENSU_CERT_FILE
example
# Command line example
sensu-agent start --cert-file /path/to/agent-1.pem

# /etc/sensu/agent.yml example
cert-file: "/path/to/agent-1.pem"
trusted-ca-file
description SSL/TLS certificate authority.
type String
default ""
environment variable SENSU_TRUSTED_CA_FILE
example
# Command line example
sensu-agent start --trusted-ca-file /path/to/trusted-certificate-authorities.pem

# /etc/sensu/agent.yml example
trusted-ca-file: "/path/to/trusted-certificate-authorities.pem"
key-file
description Path to the agent key file used in mutual TLS authentication.
type String
default ""
environment variable SENSU_KEY_FILE
example
# Command line example
sensu-agent start --key-file /path/to/agent-1-key.pem

# /etc/sensu/agent.yml example
key-file: "/path/to/agent-1-key.pem"
insecure-skip-tls-verify
description Skip SSL verification.

WARNING: This configuration flag is intended for use in development systems only. Do not use this flag in production.

type Boolean
default false
environment variable SENSU_INSECURE_SKIP_TLS_VERIFY
example
# Command line example
sensu-agent start --insecure-skip-tls-verify

# /etc/sensu/agent.yml example
insecure-skip-tls-verify: true

require-fips
description Require Federal Information Processing Standard (FIPS) support in OpenSSL. Logs an error at Sensu agent startup if true but OpenSSL is not running in FIPS mode.

NOTE: The --require-fips flag is only available within the Linux amd64 OpenSSL-linked binary. Contact Sensu to request the builds for OpenSSL with FIPS support.

type Boolean
default false
environment variable SENSU_REQUIRE_FIPS
example
# Command line example
sensu-agent start --require-fips

# /etc/sensu/agent.yml example
require-fips: true
require-openssl
description Use OpenSSL instead of Go’s standard cryptography library. Logs an error at Sensu agent startup if true but Go’s standard cryptography library is loaded.

NOTE: The --require-openssl flag is only available within the Linux amd64 OpenSSL-linked binary. Contact Sensu to request the builds for OpenSSL with FIPS support.

type Boolean
default false
environment variable SENSU_REQUIRE_OPENSSL
example
# Command line example
sensu-agent start --require-openssl

# /etc/sensu/agent.yml example
require-openssl: true

Socket configuration flags

socket-host
description Address to bind the Sensu agent socket to.
type String
default 127.0.0.1
environment variable SENSU_SOCKET_HOST
example
# Command line example
sensu-agent start --socket-host 0.0.0.0

# /etc/sensu/agent.yml example
socket-host: "0.0.0.0"
socket-port
description Port the Sensu agent socket listens on.
type Integer
default 3030
environment variable SENSU_SOCKET_PORT
example
# Command line example
sensu-agent start --socket-port 4030

# /etc/sensu/agent.yml example
socket-port: 4030
disable-sockets
description true to disable the agent TCP and UDP event sockets. Othewise, false.
type Boolean
default false
environment variable SENSU_DISABLE_SOCKETS
example
# Command line example
sensu-agent start --disable-sockets

# /etc/sensu/agent.yml example
disable-sockets: true

StatsD configuration flags

statsd-disable
description true to disable the StatsD listener and metrics server. Otherwise, false.
type Boolean
default false
environment variable SENSU_STATSD_DISABLE
example
# Command line example
sensu-agent start --statsd-disable

# /etc/sensu/agent.yml example
statsd-disable: true
statsd-event-handlers
description List of event handlers for StatsD metrics.
type List
environment variable SENSU_STATSD_EVENT_HANDLERS
example
# Command line examples
sensu-agent start --statsd-event-handlers influxdb,opentsdb
sensu-agent start --statsd-event-handlers influxdb --statsd-event-handlers opentsdb

# /etc/sensu/agent.yml example
statsd-event-handlers:
  - influxdb
  - opentsdb
statsd-flush-interval
description Number of seconds between StatsD flushes.
type Integer
default 10
environment variable SENSU_STATSD_FLUSH_INTERVAL
example
# Command line example
sensu-agent start --statsd-flush-interval 30

# /etc/sensu/agent.yml example
statsd-flush-interval: 30
statsd-metrics-host
description Address used for the StatsD metrics server.
type String
default 127.0.0.1
environment variable SENSU_STATSD_METRICS_HOST
example
# Command line example
sensu-agent start --statsd-metrics-host 0.0.0.0

# /etc/sensu/agent.yml example
statsd-metrics-host: "0.0.0.0"
statsd-metrics-port
description Port used for the StatsD metrics server.
type Integer
default 8125
environment variable SENSU_STATSD_METRICS_PORT
example
# Command line example
sensu-agent start --statsd-metrics-port 6125

# /etc/sensu/agent.yml example
statsd-metrics-port: 6125

Allow list configuration commands

The allow list includes check and hook commands the agent can execute. Use the allow-list flag to specify the path to the yaml or json file that contains your allow list.

Use these commands to build your allow list configuration file.

exec
description Command to allow the Sensu agent to run as a check or a hook.
required true
type String
example
"exec": "/usr/local/bin/check_memory.sh"
sha512
description Checksum of the check or hook executable.
required false
type String
example
"sha512": "4f926bf4328..."
args
description Arguments for the exec command.
required true
type Array
example
"args": ["foo"]
enable_env
description true to enable environment variables. Otherwise, false.
required false
type Boolean
example
"enable_env": true

Example allow list configuration file

- exec: /usr/local/bin/check_memory.sh
  args:
  - ""
  sha512: 736ac120323772543fd3a08ee54afdd54d214e58c280707b63ce652424313ef9084ca5b247d226aa09be8f831034ff4991bfb95553291c8b3dc32cad034b4706
  enable_env: true
  foo: bar
- exec: /usr/local/bin/show_process_table.sh
  args:
  - ""
  sha512: 28d61f303136b16d20742268a896bde194cc99342e02cdffc1c2186f81c5adc53f8550635156bebeed7d87a0c19a7d4b7a690f1a337cc4737e240b62b827f78a
- exec: echo-asset.sh
  args:
  - "foo"
  sha512: cce3d16e5881ba829f271df778f9014f7c3659917f7acfd7a60a91bfcabb472eea72f9781194d310388ba046c21790364ad0308a5a897cde50022195ba90924b
[
  {
    "exec": "/usr/local/bin/check_memory.sh",
    "args": [
      ""
    ],
    "sha512": "736ac120323772543fd3a08ee54afdd54d214e58c280707b63ce652424313ef9084ca5b247d226aa09be8f831034ff4991bfb95553291c8b3dc32cad034b4706",
    "enable_env": true,
    "foo": "bar"
  },
  {
    "exec": "/usr/local/bin/show_process_table.sh",
    "args": [
      ""
    ],
    "sha512": "28d61f303136b16d20742268a896bde194cc99342e02cdffc1c2186f81c5adc53f8550635156bebeed7d87a0c19a7d4b7a690f1a337cc4737e240b62b827f78a"
  },
  {
    "exec": "echo-asset.sh",
    "args": [
      "foo"
    ],
    "sha512": "cce3d16e5881ba829f271df778f9014f7c3659917f7acfd7a60a91bfcabb472eea72f9781194d310388ba046c21790364ad0308a5a897cde50022195ba90924b"
  }
]

Configuration via environment variables

Instead of using configuration flags, you can use environment variables to configure your Sensu agent. Each agent configuration flag has an associated environment variable. You can also create your own environment variables, as long as you name them correctly and save them in the correct place. Here’s how.

  1. Create the files from which the sensu-agent service configured by our supported packages will read environment variables: /etc/default/sensu-agent for Debian/Ubuntu systems or /etc/sysconfig/sensu-agent for RHEL/CentOS systems.

    $ sudo touch /etc/default/sensu-agent
    $ sudo touch /etc/sysconfig/sensu-agent
  2. Make sure the environment variable is named correctly. All environment variables controlling Sensu configuration begin with SENSU_.

    To rename a configuration flag you wish to specify as an environment variable, prepend SENSU_, convert dashes to underscores, and capitalize all letters. For example, the environment variable for the flag api-host is SENSU_API_HOST.

    For a custom test variable, the environment variable name might be SENSU_TEST_VAR.

  3. Add the environment variable to the environment file (/etc/default/sensu-agent for Debian/Ubuntu systems or /etc/sysconfig/sensu-agent for RHEL/CentOS systems).

    In this example, the api-host flag is configured as an environment variable and set to "0.0.0.0":

    $ echo 'SENSU_API_HOST="0.0.0.0"' | sudo tee -a /etc/default/sensu-agent
    $ echo 'SENSU_API_HOST="0.0.0.0"' | sudo tee -a /etc/sysconfig/sensu-agent
  4. Restart the sensu-agent service so these settings can take effect.

    $ sudo systemctl restart sensu-agent
    $ sudo systemctl restart sensu-agent

NOTE: Sensu includes an environment variable for each agent configuration flag. They are listed in the configuration flag description tables.

Format for label and annotation environment variables

To use labels and annotations as environment variables in your check and plugin configurations, you must use a specific format when you create the SENSU_LABELS and SENSU_ANNOTATIONS environment variables.

For example, to create the labels "region": "us-east-1" and "type": "website" as an environment variable:

$ echo 'SENSU_LABELS='{"region": "us-east-1", "type": "website"}'' | sudo tee -a /etc/default/sensu-agent
$ echo 'SENSU_LABELS='{"region": "us-east-1", "type": "website"}'' | sudo tee -a /etc/sysconfig/sensu-agent

To create the annotations "maintainer": "Team A" and "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX" as an environment variable:

$ echo 'SENSU_ANNOTATIONS='{"maintainer": "Team A", "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX"}'' | sudo tee -a /etc/default/sensu-agent
$ echo 'SENSU_ANNOTATIONS='{"maintainer": "Team A", "webhook-url": "https://hooks.slack.com/services/T0000/B00000/XXXXX"}'' | sudo tee -a /etc/sysconfig/sensu-agent

Use environment variables with the Sensu agent

Any environment variables you create in /etc/default/sensu-agent (Debian/Ubuntu) or /etc/sysconfig/sensu-agent (RHEL/CentOS) will be available to check and hook commands executed by the Sensu agent. This includes your checks and plugins.

For example, if you create a SENSU_TEST_VAR variable in your sensu-agent file, it will be available to use in your check configurations as $SENSU_TEST_VAR.