- Observability: other versions:
- What is Elastic Observability?
- What’s new in 8.5
- Send data to Elasticsearch
- Spin up the Elastic Stack
- Deploy Elastic Agent to send data
- Deploy Beats to send data
- Elastic Serverless Forwarder for AWS
- Deploy serverless forwarder
- Configuration options
- Troubleshooting
- Observability overview page
- Application performance monitoring (APM)
- Log monitoring
- Infrastructure monitoring
- Uptime and synthetic monitoring
- User Experience
- Alerting
- Cases
- CI/CD observability
- Troubleshooting
- Fields reference
- Tutorials
- Monitor Amazon Web Services (AWS) with Elastic Agent
- Monitor Amazon Web Services (AWS) with Beats
- Monitor Google Cloud Platform
- Monitor a Java application
- Monitor Kubernetes
- Monitor Microsoft Azure with Elastic Agent
- Monitor Microsoft Azure with the native Azure integration
- Monitor Microsoft Azure with Beats
Define include/exclude filters
editDefine include/exclude filters
editYou can define multiple filters for inputs to include or exclude events from data ingestion.
inputs: - type: "s3-sqs" id: "arn:aws:sqs:%REGION%:%ACCOUNT%:%QUEUENAME%" include: - "[a-zA-Z]" exclude: - "skip this" - "skip also this" outputs: - type: "elasticsearch" args: elasticsearch_url: "arn:aws:secretsmanager:eu-central-1:123456789:secret:es_url" username: "arn:aws:secretsmanager:eu-west-1:123456789:secret:es_secrets:username" password: "arn:aws:secretsmanager:eu-west-1:123456789:secret:es_secrets:password" es_datastream_name: "logs-generic-default"
You can define a list of regular expressions within inputs.[].include
. If this list is populated, only messages matching any of the defined regular expressions will be forwarded to the outputs.
You can define a list of regular expressions within inputs.[].exclude
. If this list is populated, only messages not matching any of the defined regular expressions will be forwarded to the outputs i.e. every message will be forwarded to the outputs unless it matches any of the defined regular expressions.
Both config parameters are optional, and can be set independently of each other. In terms of rule precedence, the exclude filter is applied first and then the include filter, so exclude takes precedence if both are specified.
All regular expressions are case-sensitive and should follow Python’s 3.9 regular expression syntax.
Messages are scanned for terms that match the defined filters. Use the ^
(caret) special character to explicitly anchor the regex to the position before the first character of the string, and use $
to anchor at the end.
No flags are used when the regular expression is compiled. Please refer to inline flag documentation for alternative options for multiline, case-insensitive, and other matching behaviors.