Stdin input

The stdin input supports the following configuration options plus the Common options described later.

The file encoding to use for reading data that contains international characters. See the encoding names recommended by the W3C for use in HTML5.

Valid encodings:

The plain encoding is special, because it does not validate or transform any input.

A list of regular expressions to match the lines that you want Filebeat to exclude. Filebeat drops any lines that match a regular expression in the list. By default, no lines are dropped. Empty lines are ignored.

If multiline settings are also specified, each multiline message is combined into a single line before the lines are filtered by exclude_lines.

The following example configures Filebeat to drop any lines that start with DBG.

filebeat.inputs:
- type: stdin
  ...
  exclude_lines: ['^DBG']

See Regular expression support for a list of supported regexp patterns.

A list of regular expressions to match the lines that you want Filebeat to include. Filebeat exports only the lines that match a regular expression in the list. By default, all lines are exported. Empty lines are ignored.

If multiline settings also specified, each multiline message is combined into a single line before the lines are filtered by include_lines.

The following example configures Filebeat to export any lines that start with ERR or WARN:

filebeat.inputs:
- type: stdin
  ...
  include_lines: ['^ERR', '^WARN']

The following example exports all log lines that contain sometext, except for lines that begin with DBG (debug messages):

filebeat.inputs:
- type: stdin
  ...
  include_lines: ['sometext']
  exclude_lines: ['^DBG']

See Regular expression support for a list of supported regexp patterns.

The size in bytes of the buffer that each harvester uses when fetching a file. The default is 16384.

The maximum number of bytes that a single log message can have. All bytes after max_bytes are discarded and not sent. This setting is especially useful for multiline log messages, which can get large. The default is 10MB (10485760).

These options make it possible for Filebeat to decode logs structured as JSON messages. Filebeat processes the logs line by line, so the JSON decoding only works if there is one JSON object per line.

The decoding happens before line filtering and multiline. You can combine JSON decoding with filtering and multiline if you set the message_key option. This can be helpful in situations where the application logs are wrapped in JSON objects, as with like it happens for example with Docker.

Example configuration:

json.keys_under_root: true
json.add_error_key: true
json.message_key: log

You must specify at least one of the following settings to enable JSON parsing mode:

Options that control how Filebeat deals with log messages that span multiple lines. See Manage multiline messages for more information about configuring multiline options.

The following configuration options are supported by all inputs.

Use the enabled option to enable and disable inputs. By default, enabled is set to true.

A list of tags that Filebeat includes in the tags field of each published event. Tags make it easy to select specific events in Kibana or apply conditional filtering in Logstash. These tags will be appended to the list of tags specified in the general configuration.

Example:

filebeat.inputs:
- type: stdin
  . . .
  tags: ["json"]

Optional fields that you can specify to add additional information to the output. For example, you might add fields that you can use for filtering log data. Fields can be scalar values, arrays, dictionaries, or any nested combination of these. By default, the fields that you specify here will be grouped under a fields sub-dictionary in the output document. To store the custom fields as top-level fields, set the fields_under_root option to true. If a duplicate field is declared in the general configuration, then its value will be overwritten by the value declared here.

filebeat.inputs:
- type: stdin
  . . .
  fields:
    app_id: query_engine_12

If this option is set to true, the custom fields are stored as top-level fields in the output document instead of being grouped under a fields sub-dictionary. If the custom field names conflict with other field names added by Filebeat, then the custom fields overwrite the other fields.

A list of processors to apply to the input data.

See Filter and enhance the exported data for information about specifying processors in your config.

The Ingest Node pipeline ID to set for the events generated by this input.

If this option is set to true, fields with null values will be published in the output document. By default, keep_null is set to false.

If present, this formatted string overrides the index for events from this input (for elasticsearch outputs), or sets the raw_index field of the event’s metadata (for other outputs). This string can only refer to the agent name and version and the event timestamp; for access to dynamic fields, use output.elasticsearch.index or a processor.

Example value: "%{[agent.name]}-myindex-%{+yyyy.MM.dd}" might expand to "filebeat-myindex-2019.11.01".