Logging configuration migration

edit

Logging configuration migration

edit

Prepare for logging in 8.0

edit

The entire logging system changed to offer a more consistent logging behavior with Elasticsearch, which uses the popular log4j 2 for managing logs. The current system differs from the more simplistic, deprecated system and allows you the freedom to customize what, how, and where Kibana outputs log messages. The best place to learn about the new logging system is our logging service guide. However, there are some important considerations when you are migrating from the old to the new system:

Mixing deprecated and current configurations

edit

Kibana supports mixed usage of the deprecated and current logging configuration styles, however, to reduce the risk of confusion that may arise from conflicting configurations, we recommend changing over to the new system all at once.

Rotating log files

edit

If you use log rotation, be careful when mixing deprecated and current configuration styles as it could lead to unexpected behavior. For example, one system could apply the log file rotation while the other still writes to the original file. Rather, switch over to the current system completely.

Service scripts

edit

If you are using the systemd service scripts (kibana.service), be aware that the deprecated logging.dest configuration might be set via CLI options in the kibana.service declaration. When logging.dest is set in the service scripts, Kibana will continue to write log messages to the "old" file. To switch over to the current logging system, remove the logging.dest CLI argument from /etc/systemd/system/kibana.service and run systemctl daemon-reload.

Compatibility

edit

Compatibility with the legacy logging system is assured until the end of the v7 version. All log messages handled by root context are forwarded to the legacy logging service. If you configure a custom appender for root, make sure to include the default appender in the appenders array to provide backward compatibility.

Duplicate log messages

edit

When you switch to the new logging configuration, you will start seeing duplicate log entries in both legacy and current formats. The legacy format will be removed when the default appender is no longer required. To override this behavior for specific log messages, configure an appender for the logger.

Memory consumption

edit

If you log large volumes to file, such as logging in verbose mode, be aware that by using the legacy logging system, your cluster may experience increased memory consumption. We recommend switching over to the current logging system to avoid out of memory issues. Refer to the public issue for more information.

Parameter Platform log record in pattern format Legacy Platform log record text format

@timestamp

ISO8601_TZ 2012-01-31T23:33:22.011-05:00

Absolute 23:33:22.011

logger

parent.child

['parent', 'child']

level

DEBUG

['debug']

meta

stringified JSON object {"to": "v8"}

N/A

pid

can be configured as %pid

N/A

Parameter Platform log record in json format Legacy Platform log record json format

@timestamp

ISO8601_TZ 2012-01-31T23:33:22.011-05:00

ISO8601 2012-01-31T23:33:22.011Z

logger

log.logger: parent.child

tags: ['parent', 'child']

level

log.level: DEBUG

tags: ['debug']

meta

merged in log record {... "to": "v8"}

merged in log record {... "to": "v8"}

pid

process.pid: 12345

pid: 12345

type

N/A

type: log

error

{ message, name, stack }

{ message, name, stack, code, signal }