Upgrade Sensu

To upgrade to the latest version of Sensu Go:

1. Install the latest packages or Docker image.

2. For systems that use systemd, run:

   sudo systemctl daemon-reload

3. Restart the Sensu agent:

   sudo service sensu-agent restart

4. Restart the Sensu backend:

   sudo service sensu-backend restart

5. Run a single upgrade command on one your Sensu backends to migrate the cluster:

   sensu-backend upgrade

   To skip confirmation and immediately run the upgrade command, add the --skip-confirm flag:

   sensu-backend upgrade --skip-confirm

   NOTE: If you are deploying a new Sensu cluster rather than upgrading from a previous version, you do not need to run the sensu-backend upgrade command.

6. Enter y or n to confirm if you did not add the --skip-confirm flag in step 5. Otherwise, skip this step.

7. Wait a few seconds for the upgrade command to run. You may notice some inconsistencies in your entity list until the cluster finishes upgrading. Despite this, your cluster will continue to publish standard check requests and process events.

   If you run the upgrade command more than once, it will not harm the cluster — you’ll just receive a response that the upgrade command has already been run.

   Some minor versions do not involve database-specific changes, and the sensu-backend upgrade tool will report that nothing was upgraded. Check the release notes to confirm whether a version has database-specific changes that require a backend upgrade.

8. Confirm the installed version for the agent, backend, and sensuctl:

   sensu-agent version

   sensu-backend version

   sensuctl version

Upgrade to Sensu Go 6.5.0 from any previous version

To use pipelines, you must upgrade your Sensu agents to Sensu Go 6.5.0. Agents that are not upgraded to 6.5.0 will run checks, send observability events to the backend, and use the handlers that are defined in check handlers arrays, but they will not run pipelines.

Upgrade to Sensu Go 6.4.0 from any previous version

In Sensu Go 6.4.0, we upgraded the embedded etcd version from 3.3.22 to 3.5.0. As a result, for deployments that use embedded etcd, 6.4.0 is not backward-compatible with previous versions of the Sensu backend. In addition, Sensu 6.4.0 is not backward-compatible for PostgreSQL deployments.

For embedded etcd deployments, before you upgrade to Sensu Go 6.4.0, use the etcd snapshot and restore process to create a full etcd database backup. If you use PostgreSQL, make sure to back up your PostgreSQL database also.

If you make a full etcd database backup (and a PostgreSQL database backup, if you use PostgreSQL) before upgrading to 6.4.0, you will be able to restore your pre-6.4.0 deployment if you need to revert to an earlier Sensu release.

After creating a full backup of your embedded etcd database and your PostgreSQL database, if you use PostgreSQL, you can complete the upgrade process.

CommonName deprecation in Go 1.15

Sensu Go 6.4.0 upgrades the Go version from 1.13.15 to 1.16.5. As of Go 1.15, certificates must include their CommonName (CN) as a Subject Alternative Name (SAN) field.

To prevent connection errors after upgrading to Sensu Go 6.4.0, follow Generate certificates to make sure your certificates’ SAN fields include their CNs.

Upgrade to Sensu Go 6.2.0 from any previous version

Prior to Sensu Go 6.0, sensu-backend allowed you to delete a namespace even when other resources still referenced that namespace. As of Sensu Go 6.0, it is not possible to delete namespaces that are referenced in other resources. As a result, users whose configuration predates Sensu Go 6.0 may have lingering resources, including check configurations, that reference non-existent namespaces.

Upgrading to Sensu Go 6.2.0 requires sensu-backend to upgrade check configurations. If you have check configurations that reference non-existent namespaces, the 6.2.0 upgrade operation will fail when it encounters one of these check configurations. You will receive an error message like this:

{"component":"store-providers","error":"the namespace test does not exist","level":"error","msg":"error enabling round robin scheduling,backend restart required","time":"2020-12-27T08:41:59Z"}

When this happens, the backend is effectively halted and subsequent restarts will result in the same state.

Remove checks that reference non-existent namespaces

Use the following commands with the jq utility to identify and remove checks that reference deleted namespaces before you upgrade to Sensu Go 6.2.0.

These commands use a Sensu backend running on localhost in the example URL and the environment variable $SENSU_API_KEY to represent a valid API key.

1. Get a list of existing namespaces. In this example, the existing namespaces are stage and dev.

   curl -s -H "Authorization: Key $SENSU_API_KEY" http://localhost:8080/api/core/v2/namespaces | jq '[.[].name]'
   [
      "stage",
      "dev"
   ]

2. Print the name and namespace for any checks that reference a namespace that is not specified in the jq expression on the same line. Your jq expression should include all of the namespaces you retrieved in step 1 (in this example, ["stage","dev"]).

   curl -s -H "Authorization: Key $SENSU_API_KEY" http://localhost:8080/api/core/v2/checks | jq '["stage","dev"] as $valid | .[] | select(.metadata.namespace as $in | $valid | index($in) | not) | {name: .metadata.name, namespace: .metadata.namespace}'
   {
     "name": "check-cpu",
     "namespace": "test"
   }

3. Recreate the missing test namespace so you can delete the check-cpu check.

   sensuctl namespace create test

4. Delete the check-cpu check.

   sensuctl check delete check-cpu --namespace test

5. Delete the test namespace, which is now empty after you deleted check-cpu in step 4.

   sensuctl namespace delete test

After completing these commands, you can upgrade to 6.2.0.

Upgrade to Sensu Go 6.1.0 from 6.0.0

If you are using 6.0.0 and have a large number of events in PostgreSQL, you may experience a short period of unavailability after you upgrade to 6.1.0. This pause will occur while the optimized selector information is populating during automatic database migration. It may last for a period of a few seconds to a few minutes.

This pause may extend to API request processing, so sensuctl and the web UI may also be unavailable during the migration.

Upgrade to Sensu Go 6.0 from a 5.x deployment

Before you upgrade to Sensu 6.0, use sensuctl dump to create a backup of your existing installation.

You will not be able to downgrade to a Sensu 5.x version after you upgrade your database to Sensu 6.0 after you restart the backend in the upgrade process.

Upgrade to Sensu Go 5.16.0 from any earlier version

As of Sensu Go 5.16.0, Sensu’s free entity limit is 100 entities. All commercial features are available for free in the packaged Sensu Go distribution for up to 100 entities.

When you upgrade to 5.16.0, if your existing unlicensed instance has more than 100 entities, Sensu will continue to monitor those entities. However, if you try to create any new entities via the HTTP API or sensuctl, you will receive the following message:

This functionality requires a valid Sensu Go license with a sufficient entity limit. To get a valid license file, arrange a trial, or increase your entity limit, contact Sales.

Connections from new agents will fail and result in a log message like this:

{"component":"agent","error":"handshake failed with status 402","level":"error","msg":"reconnection attempt failed","time":"2019-11-20T05:49:24-07:00"}

In the web UI, you will receive the following message when you reach the 100-entity limit:

If your Sensu instance includes more than 100 entities, contact Sales to learn how to upgrade your installation and increase your limit. Read our blog announcement for more information about our usage policy.

Upgrade Sensu clusters from 5.7.0 or earlier to 5.8.0 or later

Due to updates to etcd serialization, you must shut down Sensu clusters with multiple backend nodes while upgrading from Sensu Go 5.7.0 or earlier to 5.8.0 or later. Read the backend reference for more information about stopping and starting backends.

Upgrade Sensu backend binaries to 5.1.0

For Sensu backend binaries, the default state-dir in 5.1.0 is now /var/lib/sensu/sensu-backend instead of /var/lib/sensu. To upgrade your Sensu backend binary to 5.1.0, first download the latest version. Then, make sure the /etc/sensu/backend.yml configuration file specifies a state-dir. To continue using /var/lib/sensu as the state-dir to store backend data, add the following configuration to /etc/sensu/backend.yml:

state-dir: "/var/lib/sensu"

Then restart the backend:

sudo service sensu-backend restart