IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› ›

Logstash 5.3.0 Release Notes

edit

Logstash 5.3.0 Release Notes

edit

Persistent queues:
- Changed the default queue location on disk to include the pipeline’s ID in the path hierarchy. By default, the queue is now created under <path.data>/queue/main. This breaking change was made to accommodate an upcoming feature where multiple, isolated pipelines could be run on the same Logstash instance.
- Added a recovery process that runs during Logstash startup to recover data that has been written to the persistent queue, but not yet checkpointed. This is useful in situations where the input has written data to the queue, but Logstash crashed before writing to the checkpoint file.
- Added exclusive access to the persistent queue on disk, as defined by the path.queue setting. Using a file lock guards against corruption by ensuring that only a single Logstash instance has access to write to the queue on the same path. (Issue 6604).
- You can now safely reload the pipeline config when using persistent queues. Previously, reloading the config could result in data corruption. In 5.3, the reload sequence has been changed to reliably shut down the first pipeline before a new one is started with the same settings.
- Fixed an issue where Logstash would stop accepting new events when queue capacity is reached even though events were successfully acknowledged (Issue 6626).
Fixed a warning message when --config.debug is used with --log.level=debug (Issue 6256).

Input Plugins

edit

S3:

We now include the S3 key information in the metadata (Issue 105).

Unix:

The host and path fields are no longer overwritten if they are already provided by add_field config.

Filter Plugins

edit

KV:

Breaking: The trim and trimkey options are renamed to trim_value and trim_key respectively (Issue 10).
trim_value only removes the specified leading and trailing characters from the value. Similarly, trim_key only removes the specified leading and trailing characters from the key (Issue 10).
Added new options remove_char_value and remove_char_key to remove the specified characters from keys (or values) regardless of where these characters are found (Issue 10).

Grok:

Added an option to define custom patterns using pattern_definitions configuration.

Output Plugins

edit

S3:

Fixed to use the correct signature_version for the SDK v2 library (Issue 129).
Fixed an issue which resulted in uploading empty files to S3 when using gzip compression (Issue 95).

CSV:

Updated to work with the 5.0 event API and threading contracts (Issue 10).

« Logstash 5.3.1 Release Notes