Upgrade Sensu

Upgrade to Sensu Go 6.2.0 from any previous version

Prior to Sensu Go 6.0, sensu-backend allowed you to delete a namespace even when other resources still referenced that namespace. As of Sensu Go 6.0, it is not possible to delete namespaces that are referenced in other resources. As a result, users whose configuration predates Sensu Go 6.0 may have lingering resources, including check configurations, that reference non-existent namespaces.

Upgrading to Sensu Go 6.2.0 requires sensu-backend to upgrade check configurations. If you have check configurations that reference non-existent namespaces, the 6.2.0 upgrade operation will fail when it encounters one of these check configurations. You will see an error message like this:

{"component":"store-providers","error":"the namespace test does not exist","level":"error","msg":"error enabling round robin scheduling,backend restart required","time":"2020-12-27T08:41:59Z"}

When this happens, the backend is effectively halted and subsequent restarts will result in the same state.

Remove checks that reference non-existent namespaces

Use the following commands with the jq utility to identify and remove checks that reference deleted namespaces before you upgrade to Sensu Go 6.2.0.

These commands use a Sensu backend running on localhost in the example URL and the environment variable $SENSU_API_KEY to represent a valid API key.

1. Get a list of existing namespaces. In this example, the existing namespaces are stage and dev.

   $ curl -s -H "Authorization: Key $SENSU_API_KEY" http://localhost:8080/api/core/v2/namespaces | jq '[.[].name]'
   [
      "stage",
      "dev"
   ]

2. Print the name and namespace for any checks that reference a namespace that is not specified in the jq expression on the same line. Your jq expression should include all of the namespaces you retrieved in step 1 (in this example, ["stage","dev"]).

   $ curl -s -H "Authorization: Key $SENSU_API_KEY" http://localhost:8080/api/core/v2/checks | jq '["stage","dev"] as $valid | .[] | select(.metadata.namespace as $in | $valid | index($in) | not) | {name: .metadata.name, namespace: .metadata.namespace}'
   {
     "name": "check-cpu",
     "namespace": "test"
   }

3. Recreate the missing test namespace so you can delete the check-cpu check.

   $ sensuctl namespace create test

4. Delete the check-cpu check.

   $ sensuctl check delete check-cpu --namespace test

5. Delete the test namespace, which is now empty after you deleted check-cpu in step 4.

   $ sensuctl namespace delete test

After completing these commands, you can upgrade to 6.2.0.

Upgrade to Sensu Go 6.1.0 from 6.0.0

To upgrade to Sensu Go 6.1.0 from version 6.0.0 or later, install the latest packages and restart the services.

# Restart the Sensu agent
sudo service sensu-agent restart

# Restart the Sensu backend
sudo service sensu-backend restart

To confirm the installed version, run sensu-agent version, sensu-backend version, and sensuctl version.

If you have a large number of events in PostgreSQL, you may experience a short period of unavailability after you upgrade to 6.1.0. This pause will occur while the optimized selector information is populating during automatic database migration. It may last for a period of a few seconds to a few minutes.

This pause may extend to API request processing, so sensuctl and the web UI may also be unavailable during the migration.

Upgrade to Sensu Go 6.0 from a 5.x deployment

To upgrade your Sensu Go 5.x deployment to 6.0:

1. Install the 6.0 packages or Docker image.

2. Restart the services.

   NOTE: For systems that use systemd, run sudo systemctl daemon-reload before restarting the services.

   # Restart the Sensu agent
   sudo service sensu-agent restart
   
   # Restart the Sensu backend
   sudo service sensu-backend restart

3. Run a single upgrade command on one your Sensu backends to migrate the cluster:

   sensu-backend upgrade

   • Add the --skip-confirm flag to skip the confirmation in step 4 and immediately run the upgrade command.

   sensu-backend upgrade --skip-confirm

   NOTE: If you are deploying a new Sensu 6.0 cluster rather than upgrading from 5.x, you do not need to run the sensu-backend upgrade command.

4. Enter y or n to confirm if you did not add the --skip-confirm flag. Otherwise, skip this step.

5. Wait a few seconds for the upgrade command to run.

You may notice some inconsistencies in your entity list until the cluster finishes upgrading. Despite this, your cluster will continue to publish standard check requests and process events.

If you run the upgrade command more than once, it will not harm the cluster — you’ll just see a response that the upgrade command has already been run.

Upgrade to the latest 5.x version of Sensu Go from 5.0.0 or later

To upgrade to the latest version of Sensu Go from version 5.0.0 or later, install the latest packages.

Then, restart the services.

# Restart the Sensu agent
sudo service sensu-agent restart

# Restart the Sensu backend
sudo service sensu-backend restart

Use the version command to determine the installed version using the sensu-agent, sensu-backend, and sensuctl tools. For example, sensu-backend version.

Upgrade to Sensu Go 5.16.0 from any earlier version

As of Sensu Go 5.16.0, Sensu’s free entity limit is 100 entities. All commercial features are available for free in the packaged Sensu Go distribution up to an entity limit of 100.

When you upgrade to 5.16.0, if your existing unlicensed instance has more than 100 entities, Sensu will continue to monitor those entities. However, if you try to create any new entities via the HTTP API or sensuctl, you will see the following message:

This functionality requires a valid Sensu Go license with a sufficient entity limit. To get a valid license file, arrange a trial, or increase your entity limit, contact Sales.

Connections from new agents will fail and result in a log message like this:

{"component":"agent","error":"handshake failed with status 402","level":"error","msg":"reconnection attempt failed","time":"2019-11-20T05:49:24-07:00"}

In the web UI, you will see the following message when you reach the 100-entity limit:

If your Sensu instance includes more than 100 entities, contact Sales to learn how to upgrade your installation and increase your limit. See our blog announcement for more information about our usage policy.

Upgrade Sensu clusters from 5.7.0 or earlier to 5.8.0 or later

Due to updates to etcd serialization, you must shut down Sensu clusters with multiple backend nodes while upgrading from Sensu Go 5.7.0 or earlier to 5.8.0 or later. See the backend reference for more information about stopping and starting backends.

Upgrade Sensu backend binaries to 5.1.0

For Sensu backend binaries, the default state-dir in 5.1.0 is now /var/lib/sensu/sensu-backend instead of /var/lib/sensu. To upgrade your Sensu backend binary to 5.1.0, first download the latest version. Then, make sure the /etc/sensu/backend.yml configuration file specifies a state-dir. To continue using /var/lib/sensu as the state-dir, add the following configuration to /etc/sensu/backend.yml.

# /etc/sensu/backend.yml configuration to store backend data at /var/lib/sensu
state-dir: "/var/lib/sensu"

Then restart the backend.