Migrate from Sensu Core to Sensu Go

This guide includes general information for migrating your Sensu instance from Sensu Core 1.x to Sensu Go. For instructions and tools to help you translate your Sensu configuration from Sensu Core 1.x to Sensu Go, see the Sensu Translator project and our blog post about check configuration upgrades with the Sensu Go sandbox.

Sensu Go includes important changes to all parts of Sensu: architecture, installation, resource definitions, the observation data (event) model, check dependencies, filter evaluation, and more. Sensu Go also includes many powerful commercial features to make monitoring easier to build, scale, and offer as a self-service tool to your internal customers.

Sensu Go is available for RHEL/CentOS, Debian, Ubuntu, and Docker. The Sensu Go agent is also available for Windows. Configuration management integrations for Sensu Go are available for Ansible, Chef, and Puppet.

IMPORTANT: To install Sensu Go alongside your current Sensu instance, you must upgrade to at least Sensu Core 1.7.0.

Aside from this migration guide, these resources can help you migrate from Sensu Core to Sensu Go:

  • Sensu Community Slack: Join hundreds of other Sensu users in our Community Slack, where you can ask questions and benefit from tips others picked up during their own Sensu Go migrations.
  • Sensu Community Forum: Drop a question in our dedicated category for migrating to Go.
  • Sensu Go Sandbox: Download the sandbox and try out some monitoring workflows with Sensu Go.
  • Sensu Translator: Use this command-line tool to generate Sensu Go configurations from your Sensu Core config files.

We also offer commercial support and professional services packages to help with your Sensu Go migration.

Packaging

Sensu Go is provided as three packages: sensu-go-backend, sensu-go-agent, and sensu-go-cli (sensuctl). This is a fundamental change in Sensu terminology from Sensu Core: the server is now the backend.

Clients are represented within Sensu Go as abstract entities that can describe a wider range of system components such as network gear, a web server, or a cloud resource. Entities include agent entities that run a Sensu agent and the familiar proxy entities.

The glossary includes more information about new terminology in Sensu Go.

Architecture

The external RabbitMQ transport and Redis datastore in Sensu Core 1.x are replaced with an embedded transport and etcd datastore in Sensu Go.

Single Sensu Go backend or standalone architecture

Single Sensu Go backend or standalone architecture

In Sensu Go, the Sensu backend and agent are configured with YAML files or the sensu-backend or sensu-agent command line tools rather than JSON files. Sensu checks and pipeline elements are configured via the API or sensuctl tool in Sensu Go instead of JSON files.

The Sensu backend is powered by an embedded transport and etcd datastore and gives you flexible, automated workflows to route metrics and alerts. Sensu backends require persistent storage for their embedded database, disk space for local dynamic runtime asset caching, and several exposed ports:

  • 2379 (gRPC) Sensu storage client: Required for Sensu backends using an external etcd instance
  • 2380 (gRPC) Sensu storage peer: Required for other Sensu backends in a cluster
  • 3000 (HTTP/HTTPS) Sensu web UI: Required for all Sensu backends using a Sensu web UI
  • 8080 (HTTP/HTTPS) Sensu API: Required for all users accessing the Sensu API
  • 8081 (WS/WSS) Agent API: Required for all Sensu agents connecting to a Sensu backend

Sensu agents are lightweight clients that run on the infrastructure components you want to monitor. Agents automatically register with Sensu as entities and are responsible for creating check and metric events to send to the backend event pipeline.

The Sensu agent uses:

  • 3030 (TCP/UDP) Sensu agent socket: Required for Sensu agents using the agent socket
  • 3031 (HTTP) Sensu agent API: Required for all users accessing the agent API
  • 8125 (UDP) StatsD listener: Required for all Sensu agents using the StatsD listener

The agent TCP and UDP sockets are deprecated in favor of the agent API.

Agents that use Sensu dynamic runtime assets require some disk space for a local cache.

See the backend, agent, and sensuctl reference docs for more information.

Entities

“Clients” are represented within Sensu Go as abstract “entities” that can describe a wider range of system components (for example, network gear, a web server, or a cloud resource). Entities include agent entities, which are entities running a Sensu agent, and the familiar proxy entities. See the entity reference and the guide to monitoring external resources for more information.

Checks

Standalone checks are not supported in Sensu Go, although you can achieve similar functionality with role-based access control (RBAC), dynamic runtime assets, and entity subscriptions. There are also a few changes to check definitions in Sensu Go. The stdin check attribute is not supported in Sensu Go, and Sensu Go does not try to run a “default” handler when executing a check without a specified handler. In addition, check subdues are not available in Sensu Go.

Check hooks are a resource type in Sensu Go: you can create, manage, and reuse hooks independently of check definitions. You can also execute multiple hooks for any given response code.

Events

In Sensu Go, all check results are considered events and are processed by event handlers. You can use the built-in incidents filter to recreate the Sensu Core 1.x behavior in which only check results with a non-zero status are considered events.

Handlers

Transport handlers are not supported by Sensu Go, but you can create similar functionality with a pipe handler that connects to a message bus and injects event data into a queue.

Filters

Sensu Go includes three new built-in event filters: only-incidents, only-metrics, and allow-silencing. Sensu Go does not include a built-in check dependencies filter or a filter-when feature.

Ruby eval logic from Sensu Core 1.x is replaced with JavaScript expressions in Sensu Go, opening up powerful ways to filter events based on occurrences and other event attributes. As a result, Sensu Go does not include the built-in occurrence-based event filter in Sensu Core 1.x, which allowed you to control the number of duplicate events that reached the handler. You can replicate the occurrence-based filter’s functionality with Sensu Go’s repeated events filter definition.

Fatigue check filter

For Sensu Go users, we recommend the fatigue check filter, a JavaScript implementation of the occurrences filter from Sensu 1.x. This filter looks for check and entity annotations in each event it receives and uses the values of those annotations to configure the filter’s behavior on a per-event basis.

The Sensu Translator version 1.1.0 retrieves occurrence and refresh values from a Sensu Core 1.x check definition and outputs them as annotations in a Sensu Go check definition, compatible with the fatigue check filter.

However, the Sensu Translator doesn’t automatically add the fatigue check filter dynamic runtime asset or the filter configuration you need to run it. To use the fatigue check filter dynamic runtime asset, you must register it, create a correctly configured event filter definition, and add the event filter to the list of filters on applicable handlers.

Dynamic runtime assets

The sensu-install tool in Sensu Core 1.x is replaced by assets in Sensu Go. Dynamic runtime assets are shareable, reusable packages that make it easier to deploy Sensu plugins.

You can still install Sensu Community plugins in Ruby via sensu-install by installing sensu-plugins-ruby. See the installing plugins guide for more information.

Role-based access control (RBAC)

Role-based access control (RBAC) is a built-in feature of the open-source version of Sensu Go. RBAC allows you to manage and access users and resources based on namespaces, groups, roles, and bindings. To set up RBAC in Sensu Go, see the RBAC reference and Create a read-only user.

Silencing

Silencing is disabled by default in Sensu Go. You must explicitly enable silencing with the built-in not_silenced event filter.

Token substitution

The syntax for token substitution changed to double curly braces in Sensu Go (from triple colons in Sensu Core 1.x).

Aggregates

Check aggregates are supported through the Sensu Go Aggregate Check Plugin (a commercial resource).

API

In addition to the changes to resource definitions, Sensu Go includes a new, versioned API. See the API overview for more information.

Step-by-step migration instructions

Step 1: Install Sensu Go

1. Install the Sensu Go backend

The Sensu backend is available for Ubuntu/Debian, RHEL/CentOS, and Docker. See the installation guide to install, configure, and start the Sensu backend according to your deployment strategy.

2. Log in to the Sensu web UI

The Sensu Go web UI provides a unified view of your observability events with user-friendly tools to reduce alert fatigue and manage your Sensu instance. After starting the Sensu backend, open the web UI by visiting http://localhost:3000. You may need to replace localhost with the hostname or IP address where the Sensu backend is running.

To log in, enter your Sensu user credentials, or use Sensu’s default admin credentials (username: admin and password: P@ssw0rd!).

3. Install sensuctl on your workstation

Sensuctl is a command line tool for managing resources within Sensu. It works by calling Sensu’s HTTP API to create, read, update, and delete resources, events, and entities. Sensuctl is available for Linux, Windows, and macOS. See the installation guide to install and configure sensuctl.

4. Set up Sensu users

Role-based access control (RBAC) is a built-in feature of the open-source version of Sensu Go. RBAC allows you to manage and access users and resources based on namespaces, groups, roles, and bindings. To set up RBAC in Sensu Go, see the RBAC reference and Create a read-only user.

In Sensu Go, namespaces partition resources within a Sensu instance. Sensu Go entities, checks, handlers, and other namespaced resources belong to a single namespace. The Sensu translator places all translated resources into the default namespace — we’ll use the translater in a moment.

In addition to built-in RBAC, Sensu Go’s commercial features include support for authentication using Microsoft Active Directory (AD) and standards-compliant Lightweight Directory Access Protocol tools like OpenLDAP.

5. Install agents

The Sensu agent is available for Ubuntu/Debian, RHEL/CentOS, Windows, and Docker. See the installation guide to install, configure, and start Sensu agents.

If you’re doing a side-by-side migration, add api-port (default: 3031) and socket-port (default: 3030) to your agent configuration. This prevents the Sensu Go agent API and socket from conflicting with the Sensu Core client API and socket. You can also disable these features in the agent configuration using the disable-socket and disable-api flags.

# agent configuration: /etc/sensu.agent.yml
...
api-port: 4041
socket-port: 4030
...

Sensu should now be installed and functional. The next step is to translate your Sensu Core configuration to Sensu Go.

Step 2: Translate your configuration

Use t Sensu Translator command line tool to transfer your Sensu Core checks, handlers, and mutators to Sensu Go.

1. Run the translator

Install and run the translator.

# Install dependencies
yum install -q -y rubygems ruby-devel

# Install sensu-translator
gem install sensu-translator

# Translate all config in /etc/sensu/conf.d to Sensu Go and output to /sensu_config_translated
# Option: translate your config in sections according to resource type
sensu-translator -d /etc/sensu/conf.d -o /sensu_config_translated

If translation is successful, you should see a few callouts followed by DONE!.

Sensu 1.x filter translation is not yet supported
Unable to translate Sensu 1.x filter: only_production {:attributes=>{:check=>{:environment=>"production"}}}
DONE!

Combine your config into a sensuctl-readable format.

NOTE: for use with sensuctl create, do not use a comma between resource objects in Sensu Go resource definitions in JSON format.

find sensu_config_translated/ -name '*.json' -exec cat {} \; > sensu_config_translated_singlefile.json

Most attributes are ready to use as-is, but you’ll need to adjust your Sensu Go configuration manually to migrate some of Sensu’s features.

NOTE: To streamline a comparison of your Sensu Core configuration with your Sensu Go configuration, output your current Sensu Core configuration using the API: curl -s http://127.0.0.1:4567/settings | jq . > sensu_config_original.json

2. Translate checks

Review your Sensu Core check configuration for the following attributes, and make the corresponding updates to your Sensu Go configuration.

Core attribute Manual updates required in Sensu Go config
::: foo ::: Update the syntax for token substitution from triple colons to double curly braces. For example: {{ foo }}
stdin: true No updates required. Sensu Go checks accept data on stdin by default.
handlers: default Sensu Go does not have a default handler. Create a handler named default to continue using this pattern.
subdues Check subdues are not available in Sensu Go.
standalone: true Standalone checks are not supported in Sensu Go, although you can achieve similar functionality using role-based access control, dynamic runtime assets, and entity subscriptions. The translator assigns all Core standalone checks to a standalone subscription in Sensu Go. Configure one or more Sensu Go agents with the standalone subscription to execute formerly standalone checks.
metrics: true See the translate metric checks section.
proxy_requests See the translate proxy requests section.
subscribers: roundrobin... Remove roundrobin from the subscription name, and add the round_robin check attribute set to true.
aggregate Check aggregates are supported through the commercial Sensu Go Aggregate Check Plugin.
hooks See the translate hooks section.
dependencies Check dependencies are not available in Sensu Go.

PRO TIP: When using token substitution in Sensu Go and accessing labels or annotations that include . (for example: sensu.io.json_attributes), use the index function. For example, {{index .annotations "web_url"}} substitutes the value of the web_url annotation; {{index .annotations "production.ID"}} substitutes the value of the production.ID annotation.

Translate metric checks

The Sensu Core type: metric attribute is not part of the Sensu Go check spec, so you’ll need to adjust it manually. Sensu Core checks could be configured as type: metric, which told Sensu to always handle the check regardless of the check status output. This allowed Sensu Core to process output metrics via a handler even when the check status was not in an alerting state.

Sensu Go treats output metrics as first-class objects, so you can process check status as well as output metrics via different event pipelines. See the guide to metric output to update your metric checks with the output_metric_handlers and output_metric_format attributes and use output_metric_tags to enrich extracted metrics output.

Translate proxy requests and proxy entities

See the guide to monitoring external resources to re-configure proxy_requests attributes and update your proxy check configuration. See the entity reference to re-create your proxy client configurations as Sensu Go proxy entities.

Translate hooks

Check hooks are now a resource type in Sensu Go, so you can create, manage, and reuse hooks independently of check definitions. You can also execute multiple hooks for any given response code. See the guide and hooks reference docs to re-create your Sensu Core hooks as Sensu Go hook resources.

Custom attributes

Custom check attributes are not supported in Sensu Go. Instead, Sensu Go allows you to add custom labels and annotations to entities, checks, dynamic runtime assets, hooks, filters, mutators, handlers, and silences. See the metadata attributes section in the reference documentation for more information about using labels and annotations (for example, metadata attributes for entities).

The Sensu Translator stores all check extended attributes in the check metadata annotation named sensu.io.json_attributes. See the check reference for more information about using labels and annotations in check definitions.

3. Translate event filters

Ruby eval logic used in Sensu Core filters is replaced with JavaScript expressions in Sensu Go, opening up powerful possibilities to combine filters with filter dynamic runtime assets. As a result, you’ll need to rewrite your Sensu Core filters in Sensu Go format.

First, review your Core handlers to identify which filters are being used. Then, follow the filter reference and guide to using filters to re-write your filters using Sensu Go expressions and event data. Check out the blog post on filters for a deep dive into Sensu Go filter capabilities.

# Sensu Core hourly filter
{
  "filters": {
    "recurrences": {
      "attributes": {
        "occurrences": "eval: value == 1 || value % 60 == 0"
      }
    }
  }
}

# Sensu Go hourly filter
  {
    "metadata": {
      "name": "hourly",
      "namespace": "default"
    },
    "action": "allow",
    "expressions": [
      "event.check.occurrences == 1 || event.check.occurrences % (3600 / event.check.interval) == 0"
    ],
    "runtime_assets": null
  }

4. Translate handlers

In Sensu Go, all check results are considered events and are processed by event handlers. Use the built-in is_incident filter to recreate the Sensu Core behavior, in which only check results with a non-zero status are considered events.

NOTE: Silencing is disabled by default in Sensu Go and must be explicitly enabled using the built-in not_silenced filter. Add the not_silenced filter to any handlers for which you want to enable Sensu’s silencing feature.

Review your Sensu Core check configuration for the following attributes, and make the corresponding updates to your Sensu Go configuration.

Core attribute Manual updates required in Sensu Go config
filters: occurrences The built-in occurrences filter in Sensu Core is not available in Sensu Go, but you can replicate its functionality with the sensu-go-fatigue-check-filter asset.
type: transport Transport handlers are not supported in Sensu Go, but you can create similar functionality with a pipe handler that connects to a message bus and injects event data into a queue.
filters: check_dependencies Sensu Go does not include a built-in check dependencies filter.
severities Severities are not available in Sensu Go.
handle_silenced Silencing is disabled by default in Sensu Go and must be explicitly enabled using the built-in not_silenced filter.
handle_flapping All check results are considered events in Sensu Go and are processed by event handlers.

5. Upload your config to your Sensu Go instance

After you review your translated configuration, make any necessary updates, and add resource definitions for any filters and entities you want to migrate, you can upload your Sensu Go config using sensuctl.

sensuctl create --file /path/to/config.json

PRO TIP: sensuctl create (and sensuctl delete) are powerful tools to help you manage your Sensu configs across namespaces. See the sensuctl reference for more information.

Access your Sensu Go config using the Sensu API.

# Set up a local API testing environment by saving your Sensu credentials
# and token as environment variables. Requires curl and jq.
export SENSU_USER=admin && SENSU_PASS=P@ssw0rd!

export SENSU_TOKEN=`curl -XGET -u "$SENSU_USER:$SENSU_PASS" -s http://localhost:8080/auth | jq -r ".access_token"`

# Return list of all configured checks
curl -H "Authorization: Bearer $SENSU_TOKEN" http://127.0.0.1:8080/api/core/v2/namespaces/default/checks

# Return list of all configured handlers
curl -H "Authorization: Bearer $SENSU_TOKEN" http://127.0.0.1:8080/api/core/v2/namespaces/default/handlers

You can also access your Sensu Go configuration in JSON or YAML using sensuctl. For example, sensuctl check list --format json. Run sensuctl help to see available commands. For more information about sensuctl’s output formats (json, wrapped-json, and yaml), see the sensuctl reference.

Step 3: Translate plugins and register dynamic runtime assets

Sensu plugins

Within the Sensu Plugins org, see individual plugin READMEs for compatibility status with Sensu Go. For handler and mutators plugins, see the Sensu plugins README to map event data to the Sensu Go event format. This allows you to use Sensu plugins for handlers and mutators with Sensu Go without re-writing them.

To re-install Sensu plugins onto your Sensu Go agent nodes (check plugins) and backend nodes (mutator and handler plugins), see the guide to installing the sensu-install tool for use with Sensu Go.

Sensu Go dynamic runtime assets

The sensu-install tool in Sensu Core 1.x is replaced by dynamic runtime assets in Sensu Go. Dynamic runtime assets are shareable, reusable packages that make it easier to deploy Sensu plugins.

Although dynamic runtime assets are not required to run Sensu Go, we recommend using assets to install plugins where possible. You can still install Sensu Community plugins in Ruby via sensu-install by installing sensu-plugins-ruby. See the installing plugins guide for more information.

Sensu supports dynamic runtime assets for checks, filters, mutators, and handlers. Discover, download, and share dynamic runtime assets with Bonsai, the Sensu asset index.

To create your own dynamic runtime assets, see the asset reference and guide to sharing an asset on Bonsai. To contribute to converting a Sensu plugin to a dynamic runtime asset, see the Discourse post.

Step 4: Sunset your Sensu Core instance

When you’re ready to sunset your Sensu Core instance, see the platform documentation to stop the Sensu Core services. You may also want to re-install the sensu-install tool using the sensu-plugins-ruby package.