Upgrade Sensu
Upgrade to Sensu Go 6.2.0 from any previous version
Prior to Sensu Go 6.0, sensu-backend allowed you to delete a namespace even when other resources still referenced that namespace. As of Sensu Go 6.0, it is not possible to delete namespaces that are referenced in other resources. As a result, users whose configuration predates Sensu Go 6.0 may have lingering resources, including check configurations, that reference non-existent namespaces.
Upgrading to Sensu Go 6.2.0 requires sensu-backend to upgrade check configurations. If you have check configurations that reference non-existent namespaces, the 6.2.0 upgrade operation will fail when it encounters one of these check configurations. You will see an error message like this:
{"component":"store-providers","error":"the namespace test does not exist","level":"error","msg":"error enabling round robin scheduling,backend restart required","time":"2020-12-27T08:41:59Z"}
When this happens, the backend is effectively halted and subsequent restarts will result in the same state.
Remove checks that reference non-existent namespaces
Use the following commands with the jq utility to identify and remove checks that reference deleted namespaces before you upgrade to Sensu Go 6.2.0.
NOTE: If you have already upgraded to Sensu Go 6.2.0, you can work around this issue by temporarily reverting your Sensu instance to Sensu Go 6.1.4. Then, recreate the missing namespaces referenced in your check configurations and upgrade again to 6.2.0.
These commands use a Sensu backend running on localhost
in the example URL and the environment variable $SENSU_API_KEY to represent a valid API key.
-
Get a list of existing namespaces. In this example, the existing namespaces are
stage
anddev
.$ curl -s -H "Authorization: Key $SENSU_API_KEY" http://localhost:8080/api/core/v2/namespaces | jq '[.[].name]' [ "stage", "dev" ]
-
Print the name and namespace for any checks that reference a namespace that is not specified in the jq expression on the same line. Your jq expression should include all of the namespaces you retrieved in step 1 (in this example,
["stage","dev"]
).$ curl -s -H "Authorization: Key $SENSU_API_KEY" http://localhost:8080/api/core/v2/checks | jq '["stage","dev"] as $valid | .[] | select(.metadata.namespace as $in | $valid | index($in) | not) | {name: .metadata.name, namespace: .metadata.namespace}' { "name": "check-cpu", "namespace": "test" }
-
Recreate the missing
test
namespace so you can delete thecheck-cpu
check.$ sensuctl namespace create test
-
Delete the
check-cpu
check.$ sensuctl check delete check-cpu --namespace test
-
Delete the
test
namespace, which is now empty after you deletedcheck-cpu
in step 4.$ sensuctl namespace delete test
After completing these commands, you can upgrade to 6.2.0.
Upgrade to Sensu Go 6.1.0 from 6.0.0
To upgrade to Sensu Go 6.1.0 from version 6.0.0 or later, install the latest packages and restart the services.
NOTE: For systems that use systemd
, run sudo systemctl daemon-reload
before restarting the services.
# Restart the Sensu agent
sudo service sensu-agent restart
# Restart the Sensu backend
sudo service sensu-backend restart
To confirm the installed version, run sensu-agent version
, sensu-backend version
, and sensuctl version
.
If you have a large number of events in PostgreSQL, you may experience a short period of unavailability after you upgrade to 6.1.0. This pause will occur while the optimized selector information is populating during automatic database migration. It may last for a period of a few seconds to a few minutes.
This pause may extend to API request processing, so sensuctl and the web UI may also be unavailable during the migration.
Upgrade to Sensu Go 6.0 from a 5.x deployment
IMPORTANT: Before you upgrade to Sensu 6.0, use sensuctl dump
to create a backup of your existing installation.
You will not be able to downgrade to a Sensu 5.x version after you upgrade your database to Sensu 6.0 in step 3 of this process.
To upgrade your Sensu Go 5.x deployment to 6.0:
-
Install the 6.0 packages or Docker image.
-
Restart the services.
NOTE: For systems that use
systemd
, runsudo systemctl daemon-reload
before restarting the services.# Restart the Sensu agent sudo service sensu-agent restart # Restart the Sensu backend sudo service sensu-backend restart
-
Run a single upgrade command on one your Sensu backends to migrate the cluster:
sensu-backend upgrade
- Add the
--skip-confirm
flag to skip the confirmation in step 4 and immediately run the upgrade command.
sensu-backend upgrade --skip-confirm
NOTE: If you are deploying a new Sensu 6.0 cluster rather than upgrading from 5.x, you do not need to run the
sensu-backend upgrade
command. - Add the
-
Enter
y
orn
to confirm if you did not add the--skip-confirm
flag. Otherwise, skip this step. -
Wait a few seconds for the upgrade command to run.
You may notice some inconsistencies in your entity list until the cluster finishes upgrading. Despite this, your cluster will continue to publish standard check requests and process events.
If you run the upgrade command more than once, it will not harm the cluster — you’ll just see a response that the upgrade command has already been run.
Upgrade to the latest 5.x version of Sensu Go from 5.0.0 or later
To upgrade to the latest version of Sensu Go from version 5.0.0 or later, install the latest packages.
Then, restart the services.
NOTE: For systems that use systemd
, run sudo systemctl daemon-reload
before restarting the services.
# Restart the Sensu agent
sudo service sensu-agent restart
# Restart the Sensu backend
sudo service sensu-backend restart
Use the version
command to determine the installed version using the sensu-agent
, sensu-backend
, and sensuctl
tools.
For example, sensu-backend version
.
Upgrade to Sensu Go 5.16.0 from any earlier version
As of Sensu Go 5.16.0, Sensu’s free entity limit is 100 entities. All commercial features are available for free in the packaged Sensu Go distribution up to an entity limit of 100.
When you upgrade to 5.16.0, if your existing unlicensed instance has more than 100 entities, Sensu will continue to monitor those entities. However, if you try to create any new entities via the HTTP API or sensuctl, you will see the following message:
This functionality requires a valid Sensu Go license with a sufficient entity limit. To get a valid license file, arrange a trial, or increase your entity limit, contact Sales.
Connections from new agents will fail and result in a log message like this:
{"component":"agent","error":"handshake failed with status 402","level":"error","msg":"reconnection attempt failed","time":"2019-11-20T05:49:24-07:00"}
In the web UI, you will see the following message when you reach the 100-entity limit:
If your Sensu instance includes more than 100 entities, contact Sales to learn how to upgrade your installation and increase your limit. See our blog announcement for more information about our usage policy.
Upgrade Sensu clusters from 5.7.0 or earlier to 5.8.0 or later
NOTE: This section applies only to Sensu clusters with multiple backend nodes.
Due to updates to etcd serialization, you must shut down Sensu clusters with multiple backend nodes while upgrading from Sensu Go 5.7.0 or earlier to 5.8.0 or later. See the backend reference for more information about stopping and starting backends.
Upgrade Sensu backend binaries to 5.1.0
NOTE: This section applies only to Sensu backend binaries downloaded from s3-us-west-2.amazonaws.com/sensu.io/sensu-go
, not to Sensu RPM or DEB packages.
For Sensu backend binaries, the default state-dir
in 5.1.0 is now /var/lib/sensu/sensu-backend
instead of /var/lib/sensu
.
To upgrade your Sensu backend binary to 5.1.0, first download the latest version.
Then, make sure the /etc/sensu/backend.yml
configuration file specifies a state-dir
.
To continue using /var/lib/sensu
as the state-dir
, add the following configuration to /etc/sensu/backend.yml
.
# /etc/sensu/backend.yml configuration to store backend data at /var/lib/sensu
state-dir: "/var/lib/sensu"
Then restart the backend.