Log Sensu services with systemd
By default, systems where systemd is the service manager do not write logs to /var/log/sensu/
for the sensu-agent
and the sensu-backend
services.
This guide explains how to add log forwarding from journald to syslog, have rsyslog write logging data to disk, and set up log rotation of the newly created log files.
Configure journald
To configure journald to forward logging data to syslog, modify /etc/systemd/journald.conf
to include the following line:
ForwardToSyslog=yes
Configure rsyslog
Next, set up rsyslog to write the logging data received from journald to /var/log/sensu/servicename.log
.
In this example, the sensu-backend
and sensu-agent
logging data is sent to individual files named after the service.
The sensu-backend
is not required if you’re only setting up log forwarding for the sensu-agent
service.
# For the sensu-backend service, inside /etc/rsyslog.d/99-sensu-backend.conf
if $programname == 'sensu-backend' then {
/var/log/sensu/sensu-backend.log
~
}
# For the sensu-agent service, inside /etc/rsyslog.d/99-sensu-agent.conf
if $programname == 'sensu-agent' then {
/var/log/sensu/sensu-agent.log
~
}
On Ubuntu systems, run chown -R syslog:adm /var/log/sensu
so syslog can write to that directory.
Restart rsyslog and journald to apply the new configuration:
systemctl restart systemd-journald
systemctl restart rsyslog
NOTE: Sensu log messages include the Sensu log level as part of the log data. Users with rsyslog expertise may be able to extract the log level from Sensu log messages and use rsyslog processing capabilities to separate the log messages into different files based on log level.
Set up log rotation
Set up log rotation for newly created log files to ensure logging does not fill up your disk.
These examples rotate the log files /var/log/sensu/sensu-agent.log
and /var/log/sensu/sensu-backend.log
weekly, unless the size of 100M is reached first.
The last seven rotated logs are kept and compressed, with the exception of the most recent log.
After rotation, rsyslog
is restarted to ensure logging is written to a new file and not the most recent rotated file.
# Inside /etc/logrotate.d/sensu-agent.conf
/var/log/sensu/sensu-agent.log {
daily
rotate 7
size 100M
compress
delaycompress
postrotate
/bin/systemctl restart rsyslog
endscript
}
# Inside /etc/logrotate.d/sensu-backend.conf
/var/log/sensu/sensu-backend.log {
daily
rotate 7
size 100M
compress
delaycompress
postrotate
/bin/systemctl restart rsyslog
endscript
}
You can use the following command to see what logrotate would do if it were executed now based on the above schedule and size threshold.
The -d
flag will output details, but it will not take action on the logs or execute the postrotate script:
logrotate -d /etc/logrotate.d/sensu.conf
Next steps
Sensu also offers logging of event data to a separate JSON log file as a commercial feature. See the Sensu backend reference for more information about event logging.