- Filebeat Reference: other versions:
- Filebeat overview
- Quick start: installation and configuration
- Set up and run
- Upgrade
- How Filebeat works
- Configure
- Inputs
- Modules
- General settings
- Project paths
- Config file loading
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Kibana endpoint
- Kibana dashboards
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_nomad_metadata
- add_observer_metadata
- add_process_metadata
- add_tags
- append
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_cef
- decode_csv_fields
- decode_duration
- decode_json_fields
- decode_xml
- decode_xml_wineventlog
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- move_fields
- parse_aws_vpc_flow_log
- rate_limit
- registered_domain
- rename
- replace
- script
- syslog
- timestamp
- translate_sid
- truncate_fields
- urldecode
- Autodiscover
- Internal queue
- Load balancing
- Logging
- HTTP endpoint
- Regular expression support
- Instrumentation
- Feature flags
- filebeat.reference.yml
- How to guides
- Override configuration settings
- Load the Elasticsearch index template
- Change the index name
- Load Kibana dashboards
- Load ingest pipelines
- Enrich events with geoIP information
- Deduplicate data
- Parse data using an ingest pipeline
- Use environment variables in the configuration
- Avoid YAML formatting problems
- Migrate
log
input configurations tofilestream
- Modules
- Modules overview
- ActiveMQ module
- Apache module
- Auditd module
- AWS module
- AWS Fargate module
- Azure module
- Barracuda module
- Bluecoat module
- CEF module
- Check Point module
- Cisco module
- CoreDNS module
- CrowdStrike module
- Cyberark PAS module
- Cylance module
- Elasticsearch module
- Envoyproxy Module
- F5 module
- Fortinet module
- Google Cloud module
- Google Workspace module
- HAproxy module
- IBM MQ module
- Icinga module
- IIS module
- Imperva module
- Infoblox module
- Iptables module
- Juniper module
- Kafka module
- Kibana module
- Logstash module
- Microsoft module
- MISP module
- MongoDB module
- MSSQL module
- MySQL module
- MySQL Enterprise module
- NATS module
- NetFlow module
- Netscout module
- Nginx module
- Office 365 module
- Okta module
- Oracle module
- Osquery module
- Palo Alto Networks module
- pensando module
- PostgreSQL module
- Proofpoint module
- RabbitMQ module
- Radware module
- Redis module
- Salesforce module
- Santa module
- Snort module
- Snyk module
- Sonicwall module
- Sophos module
- Squid module
- Suricata module
- System module
- Threat Intel module
- Tomcat module
- Traefik module
- Zeek (Bro) Module
- ZooKeeper module
- Zoom module
- Zscaler module
- Exported fields
- ActiveMQ fields
- Apache fields
- Auditd fields
- AWS fields
- AWS CloudWatch fields
- AWS Fargate fields
- Azure fields
- Barracuda Web Application Firewall fields
- Beat fields
- Blue Coat Director fields
- Decode CEF processor fields fields
- CEF fields
- Checkpoint fields
- Cisco fields
- Cloud provider metadata fields
- Coredns fields
- Crowdstrike fields
- CyberArk PAS fields
- CylanceProtect fields
- Docker fields
- ECS fields
- Elasticsearch fields
- Envoyproxy fields
- Big-IP Access Policy Manager fields
- Fortinet fields
- Google Cloud Platform (GCP) fields
- google_workspace fields
- HAProxy fields
- Host fields
- ibmmq fields
- Icinga fields
- IIS fields
- Imperva SecureSphere fields
- Infoblox NIOS fields
- iptables fields
- Jolokia Discovery autodiscover provider fields
- Juniper JUNOS fields
- Kafka fields
- kibana fields
- Kubernetes fields
- Log file content fields
- logstash fields
- Lumberjack fields
- Microsoft fields
- MISP fields
- mongodb fields
- mssql fields
- MySQL fields
- MySQL Enterprise fields
- NATS fields
- NetFlow fields
- Arbor Peakflow SP fields
- Nginx fields
- Office 365 fields
- Okta fields
- Oracle fields
- Osquery fields
- panw fields
- Pensando fields
- PostgreSQL fields
- Process fields
- Proofpoint Email Security fields
- RabbitMQ fields
- Radware DefensePro fields
- Redis fields
- s3 fields
- Salesforce fields
- Google Santa fields
- Snort/Sourcefire fields
- Snyk fields
- Sonicwall-FW fields
- sophos fields
- Squid fields
- Suricata fields
- System fields
- threatintel fields
- Apache Tomcat fields
- Traefik fields
- Zeek fields
- ZooKeeper fields
- Zoom fields
- Zscaler NSS fields
- Monitor
- Secure
- Troubleshoot
- Get help
- Debug
- Common problems
- Error extracting container id while using Kubernetes metadata
- Can’t read log files from network volumes
- Filebeat isn’t collecting lines from a file
- Too many open file handlers
- Registry file is too large
- Inode reuse causes Filebeat to skip lines
- Log rotation results in lost or duplicate events
- Open file handlers cause issues with Windows file rotation
- Filebeat is using too much CPU
- Dashboard in Kibana is breaking up data fields incorrectly
- Fields are not indexed or usable in Kibana visualizations
- Filebeat isn’t shipping the last line of a file
- Filebeat keeps open file handlers of deleted files for a long time
- Filebeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Dashboard could not locate the index-pattern
- High RSS memory usage due to MADV settings
- Contribute to Beats
Filebeat command reference
editFilebeat command reference
editFilebeat provides a command-line interface for starting Filebeat and performing common tasks, like testing configuration files and loading dashboards.
The command-line also supports global flags for controlling global behaviors.
Use sudo
to run the following commands if:
-
the config file is owned by
root
, or -
Filebeat is configured to capture data that requires
root
access
Some of the features described here require an Elastic license. For more information, see https://www.elastic.co/subscriptions and License Management.
Commands | |
---|---|
Exports the configuration, index template, ILM policy, or a dashboard to stdout. |
|
Shows help for any command. |
|
Manages the secrets keystore. |
|
Manages configured modules. |
|
Runs Filebeat. This command is used by default if you start Filebeat without specifying a command. |
|
Sets up the initial environment, including the index template, ILM policy and write alias, Kibana dashboards (when available), and machine learning jobs (when available). |
|
Tests the configuration. |
|
Shows information about the current version. |
Also see Global flags.
export
command
editExports the configuration, index template, ILM policy, or a dashboard to stdout. You can use this command to quickly view your configuration, see the contents of the index template and the ILM policy, or export a dashboard from Kibana.
SYNOPSIS
filebeat export SUBCOMMAND [FLAGS]
SUBCOMMANDS
-
config
-
Exports the current configuration to stdout. If you use the
-c
flag, this command exports the configuration that’s defined in the specified file. -
dashboard
-
Exports a dashboard. You can use this option to store a dashboard on disk in a module and load it automatically. For example, to export the dashboard to a JSON file, run:
filebeat export dashboard --id="DASHBOARD_ID" > dashboard.json
To find the
DASHBOARD_ID
, look at the URL for the dashboard in Kibana. By default,export dashboard
writes the dashboard to stdout. The example shows how to write the dashboard to a JSON file so that you can import it later. The JSON file will contain the dashboard with all visualizations and searches. You must load the index pattern separately for Filebeat.To load the dashboard, copy the generated
dashboard.json
file into thekibana/6/dashboard
directory of Filebeat, and runfilebeat setup --dashboards
to import the dashboard.If Kibana is not running on
localhost:5061
, you must also adjust the Filebeat configuration undersetup.kibana
. -
template
-
Exports the index template to stdout. You can specify the
--es.version
and--index
flags to further define what gets exported. Furthermore you can export the template to a file instead ofstdout
by defining a directory via--dir
.
-
ilm-policy
-
Exports the index lifecycle management policy to stdout. You can specify the
--es.version
and a--dir
to which the policy should be exported as a file rather than exporting tostdout
.
FLAGS
-
--es.version VERSION
-
When used with
template
, exports an index template that is compatible with the specified version. When used withilm-policy
, exports the ILM policy if the specified ES version is enabled for ILM. -
-h, --help
-
Shows help for the
export
command. -
--index BASE_NAME
-
When used with
template
, sets the base name to use for the index template. If this flag is not specified, the default base name isfilebeat
. -
--dir DIRNAME
-
Define a directory to which the template, pipelines, and ILM policy
should be exported to as files instead of printing them to
stdout
. -
--id DASHBOARD_ID
-
When used with
dashboard
, specifies the dashboard ID.
Also see Global flags.
EXAMPLES
filebeat export config filebeat export template --es.version 8.7.1 --index myindexname filebeat export dashboard --id="a7b35890-8baa-11e8-9676-ef67484126fb" > dashboard.json
help
command
editShows help for any command.
If no command is specified, shows help for the run
command.
SYNOPSIS
filebeat help COMMAND_NAME [FLAGS]
-
COMMAND_NAME
- Specifies the name of the command to show help for.
FLAGS
-
-h, --help
-
Shows help for the
help
command.
Also see Global flags.
EXAMPLE
filebeat help export
keystore
command
editManages the secrets keystore.
SYNOPSIS
filebeat keystore SUBCOMMAND [FLAGS]
SUBCOMMANDS
-
add KEY
-
Adds the specified key to the keystore. Use the
--force
flag to overwrite an existing key. Use the--stdin
flag to pass the value throughstdin
. -
create
-
Creates a keystore to hold secrets. Use the
--force
flag to overwrite the existing keystore. -
list
- Lists the keys in the keystore.
-
remove KEY
- Removes the specified key from the keystore.
FLAGS
-
--force
-
Valid with the
add
andcreate
subcommands. When used withadd
, overwrites the specified key. When used withcreate
, overwrites the keystore. -
--stdin
-
When used with
add
, uses the stdin as the source of the key’s value. -
-h, --help
-
Shows help for the
keystore
command.
Also see Global flags.
EXAMPLES
filebeat keystore create
filebeat keystore add ES_PWD
filebeat keystore remove ES_PWD
filebeat keystore list
See Secrets keystore for more examples.
modules
command
editManages configured modules. You can use this command to enable and disable
specific module configurations defined in the modules.d
directory. The
changes you make with this command are persisted and used for subsequent
runs of Filebeat.
To see which modules are enabled and disabled, run the list
subcommand.
SYNOPSIS
filebeat modules SUBCOMMAND [FLAGS]
SUBCOMMANDS
-
disable MODULE_LIST
- Disables the modules specified in the space-separated list.
-
enable MODULE_LIST
- Enables the modules specified in the space-separated list.
-
list
- Lists the modules that are currently enabled and disabled.
FLAGS
-
-h, --help
-
Shows help for the
modules
command.
Also see Global flags.
EXAMPLES
filebeat modules list
filebeat modules enable apache2 auditd mysql
run
command
editRuns Filebeat. This command is used by default if you start Filebeat without specifying a command.
SYNOPSIS
filebeat run [FLAGS]
Or:
filebeat [FLAGS]
FLAGS
-
-N, --N
- Disables publishing for testing purposes. This option disables all outputs except the File output.
-
--cpuprofile FILE
- Writes CPU profile data to the specified file. This option is useful for troubleshooting Filebeat.
-
-h, --help
-
Shows help for the
run
command. -
--httpprof [HOST]:PORT
- Starts an http server for profiling. This option is useful for troubleshooting and profiling Filebeat.
-
--memprofile FILE
- Writes memory profile data to the specified output file. This option is useful for troubleshooting Filebeat.
-
--modules MODULE_LIST
-
Specifies a comma-separated list of modules to run. For example:
filebeat run --modules nginx,mysql,system
Rather than specifying the list of modules every time you run Filebeat, you can use the
modules
command to enable and disable specific modules. Then when you run Filebeat, it will run any modules that are enabled. -
--once
-
When the
--once
flag is used, Filebeat starts all configured harvesters and inputs, and runs each input until the harvesters are closed. If you set the--once
flag, you should also setclose_eof
so the harvester is closed when the end of the file is reached. By default harvesters are closed afterclose_inactive
is reached. -
--system.hostfs MOUNT_POINT
-
Specifies the mount point of the host’s filesystem for use in monitoring a host.
This flag is depricated, and an alternate hostfs should be specified via the
hostfs
module config value.
Also see Global flags.
EXAMPLE
filebeat run -e
Or:
filebeat -e
setup
command
editSets up the initial environment, including the index template, ILM policy and write alias, Kibana dashboards (when available), and machine learning jobs (when available)
- The index template ensures that fields are mapped correctly in Elasticsearch. If index lifecycle management is enabled it also ensures that the defined ILM policy and write alias are connected to the indices matching the index template. The ILM policy takes care of the lifecycle of an index, when to do a rollover, when to move an index from the hot phase to the next phase, etc.
- The Kibana dashboards make it easier for you to visualize Filebeat data in Kibana.
- The machine learning jobs contain the configuration information and metadata necessary to analyze data for anomalies.
This command sets up the environment without actually running Filebeat and ingesting data. Specify optional flags to set up a subset of assets.
SYNOPSIS
filebeat setup [FLAGS]
FLAGS
-
--dashboards
- Sets up the Kibana dashboards (when available). This option loads the dashboards from the Filebeat package. For more options, such as loading customized dashboards, see Importing Existing Beat Dashboards in the Beats Developer Guide.
-
-h, --help
-
Shows help for the
setup
command. -
--modules MODULE_LIST
-
Specifies a comma-separated list of modules. Use this flag to avoid errors when
there are no modules defined in the
filebeat.yml
file. -
--pipelines
-
Sets up ingest pipelines for configured filesets. Filebeat looks for
enabled modules in the
filebeat.yml
file. If you used themodules
command to enable modules in themodules.d
directory, also specify the--modules
flag. -
--enable-all-filesets
-
Enables all modules and filesets. This is useful with
--pipelines
if you want to load all ingest pipelines. Without this option you would have to list every module with themodules
command and enable every fileset within the module with a-M
option, to load all of the ingest pipelines. -
--index-management
- Sets up components related to Elasticsearch index management including template, ILM policy, and write alias (if supported and configured).
Also see Global flags.
EXAMPLES
filebeat setup --dashboards filebeat setup --pipelines filebeat setup --pipelines --modules system,nginx,mysql filebeat setup --index-management
If you used the |
test
command
editTests the configuration.
SYNOPSIS
filebeat test SUBCOMMAND [FLAGS]
SUBCOMMANDS
-
config
- Tests the configuration settings.
-
output
- Tests that Filebeat can connect to the output by using the current settings.
FLAGS
-
-h, --help
-
Shows help for the
test
command.
Also see Global flags.
EXAMPLE
filebeat test config
version
command
editShows information about the current version.
SYNOPSIS
filebeat version [FLAGS]
FLAGS
-
-h, --help
-
Shows help for the
version
command.
Also see Global flags.
EXAMPLE
filebeat version
Global flags
editThese global flags are available whenever you run Filebeat.
-
-E, --E "SETTING_NAME=VALUE"
-
Overrides a specific configuration setting. You can specify multiple overrides. For example:
filebeat -E "name=mybeat" -E "output.elasticsearch.hosts=['http://myhost:9200']"
This setting is applied to the currently running Filebeat process. The Filebeat configuration file is not changed.
-
-M, --M "VAR_NAME=VALUE"
-
Overrides the default configuration for a Filebeat module. You can specify multiple variable overrides. For example:
filebeat -modules=nginx -M "nginx.access.var.paths=['/var/log/nginx/access.log*']" -M "nginx.access.var.pipeline=no_plugins"
-
-c, --c FILE
-
Specifies the configuration file to use for Filebeat. The file you specify
here is relative to
path.config
. If the-c
flag is not specified, the default config file,filebeat.yml
, is used. -
-d, --d SELECTORS
-
Enables debugging for the specified selectors. For the selectors, you can
specify a comma-separated
list of components, or you can use
-d "*"
to enable debugging for all components. For example,-d "publisher"
displays all the publisher-related messages. -
-e, --e
- Logs to stderr and disables syslog/file output.
-
-environment
-
For logging purposes, specifies the environment that Filebeat is running in.
This setting is used to select a default log output when no log output is configured.
Supported values are:
systemd
,container
,macos_service
, andwindows_service
. Ifsystemd
orcontainer
is specified, Filebeat will log to stdout and stderr by default. -
--path.config
- Sets the path for configuration files. See the Directory layout section for details.
-
--path.data
- Sets the path for data files. See the Directory layout section for details.
-
--path.home
- Sets the path for miscellaneous files. See the Directory layout section for details.
-
--path.logs
- Sets the path for log files. See the Directory layout section for details.
-
--strict.perms
-
Sets strict permission checking on configuration files. The default is
-strict.perms=true
. See Config file ownership and permissions for more information. -
-v, --v
- Logs INFO-level messages.
On this page