How to send alerts to Slack with handlers

What are Sensu handlers?

Sensu event handlers are actions executed by the Sensu server on events.

Why use a handler?

Handlers can be used for sending an email alert, creating or resolving an incident (in PagerDuty, for example), or storing metrics in a time-series database (InfluxDB, for example).

Using a handler to send alerts to Slack

The purpose of this guide is to help you send alerts to Slack, on the channel monitoring, by configuring a handler named slack to a check named check-cpu. If you don’t already have a check in place, this guide is a great place to start.

Registering the asset

Assets are shareable, reusable packages that make it easy to deploy Sensu plugins. In this guide, we’ll use the Sensu Slack handler asset to power a slack handler.

You can use the following sensuctl example to register the Sensu Slack handler asset for Linux AMD64, or you can download the latest asset definition for your platform from Bonsai and register the asset using sensuctl create --file filename.yml.

sensuctl asset create sensu-slack-handler --url "https://github.com/sensu/sensu-slack-handler/releases/download/1.0.3/sensu-slack-handler_1.0.3_linux_amd64.tar.gz" --sha512 "68720865127fbc7c2fe16ca4d7bbf2a187a2df703f4b4acae1c93e8a66556e9079e1270521999b5871473e6c851f51b34097c54fdb8d18eedb7064df9019adc8"

You should see a confirmation message from sensuctl.

Created

Getting a Slack webhook

If you’re already an admin of a Slack, visit https://YOUR WORKSPACE NAME HERE.slack.com/services/new/incoming-webhook and follow the steps to add the Incoming WebHooks integration, choose a channel, and save the settings. (If you’re not yet a Slack admin, start here to create a new workspace.) After saving, you’ll see your webhook URL under Integration Settings.

Creating the handler

Now we’ll use sensuctl to create a handler called slack that pipes event data to Slack using the sensu-slack-handler asset. Edit the command below to include your Slack channel and webhook URL. For more information about customizing your Sensu slack alerts, see the asset page in Bonsai.

sensuctl handler create slack \
--type pipe \
--env-vars "SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T0000/B000/XXXXXXXX" \
--command "sensu-slack-handler --channel '#monitoring'" \
--runtime-assets sensu-slack-handler

You should see a confirmation message from sensuctl.

Created

Assigning the handler to a check

With the slack handler now created, it can be assigned to a check. Here, since we want to receive Slack alerts whenever the CPU usage of our systems reach some specific thresholds, we will apply our handler to the check check-cpu.

sensuctl check set-handlers check-cpu slack

Validating the handler

It might take a few moments, once the handler is assigned to the check, for the check to be scheduled on the entities and the result sent back to Sensu backend, but once an event is handled, you should see the following message in Slack.

Otherwise, you can verify the proper behavior of this handler by using sensu-backend logs. See the troubleshooting guide for log locations by platform.

Whenever an event is being handled, a log entry is added with the message "handler":"slack","level":"debug","msg":"sending event to handler", followed by a second one with the message "msg":"pipelined executed event pipe handler","output":"","status":0.

Next steps

You now know how to apply a handler to a check and take action on events. From this point, here are some recommended resources: