- Metricbeat Reference: other versions:
- Metricbeat overview
- Quick start: installation and configuration
- Set up and run
- Upgrade Metricbeat
- How Metricbeat works
- Configure
- Modules
- General settings
- Project paths
- Config file loading
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Kibana endpoint
- Kibana dashboards
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_nomad_metadata
- add_observer_metadata
- add_process_metadata
- add_tags
- append
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_duration
- decode_json_fields
- decode_xml
- decode_xml_wineventlog
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- move_fields
- rate_limit
- registered_domain
- rename
- replace
- script
- syslog
- translate_sid
- truncate_fields
- urldecode
- Autodiscover
- Internal queue
- Logging
- HTTP endpoint
- Regular expression support
- Instrumentation
- Feature flags
- metricbeat.reference.yml
- How to guides
- Modules
- ActiveMQ module
- Aerospike module
- Airflow module
- Apache module
- AWS module
- AWS billing metricset
- AWS cloudwatch metricset
- AWS dynamodb metricset
- AWS ebs metricset
- AWS ec2 metricset
- AWS elb metricset
- AWS kinesis metricset
- AWS lambda metricset
- AWS natgateway metricset
- AWS rds metricset
- AWS s3_daily_storage metricset
- AWS s3_request metricset
- AWS sns metricset
- AWS sqs metricset
- AWS transitgateway metricset
- AWS usage metricset
- AWS vpn metricset
- AWS Fargate module
- Azure module
- Azure app_insights metricset
- Azure app_state metricset
- Azure billing metricset
- Azure compute_vm metricset
- Azure compute_vm_scaleset metricset
- Azure container_instance metricset
- Azure container_registry metricset
- Azure container_service metricset
- Azure database_account metricset
- Azure monitor metricset
- Azure storage metricset
- Beat module
- Ceph module
- Ceph cluster_disk metricset
- Ceph cluster_health metricset
- Ceph cluster_status metricset
- Ceph mgr_cluster_disk metricset
- Ceph mgr_cluster_health metricset
- Ceph mgr_osd_perf metricset
- Ceph mgr_osd_pool_stats metricset
- Ceph mgr_osd_tree metricset
- Ceph mgr_pool_disk metricset
- Ceph monitor_health metricset
- Ceph osd_df metricset
- Ceph osd_tree metricset
- Ceph pool_disk metricset
- Cloudfoundry module
- CockroachDB module
- Consul module
- Containerd module
- Coredns module
- Couchbase module
- CouchDB module
- Docker module
- Dropwizard module
- Elasticsearch module
- Elasticsearch ccr metricset
- Elasticsearch cluster_stats metricset
- Elasticsearch enrich metricset
- Elasticsearch index metricset
- Elasticsearch index_recovery metricset
- Elasticsearch index_summary metricset
- Elasticsearch ingest_pipeline metricset
- Elasticsearch ml_job metricset
- Elasticsearch node metricset
- Elasticsearch node_stats metricset
- Elasticsearch pending_tasks metricset
- Elasticsearch shard metricset
- Enterprise Search module
- Envoyproxy module
- Etcd module
- Google Cloud Platform module
- Google Cloud Platform billing metricset
- Google Cloud Platform carbon metricset
- Google Cloud Platform compute metricset
- Google Cloud Platform dataproc metricset
- Google Cloud Platform firestore metricset
- Google Cloud Platform gke metricset
- Google Cloud Platform loadbalancing metricset
- Google Cloud Platform metrics metricset
- Google Cloud Platform pubsub metricset
- Google Cloud Platform storage metricset
- Golang module
- Graphite module
- HAProxy module
- HTTP module
- IBM MQ module
- IIS module
- Istio module
- Jolokia module
- Kafka module
- Kibana module
- Kubernetes module
- Kubernetes apiserver metricset
- Kubernetes container metricset
- Kubernetes controllermanager metricset
- Kubernetes event metricset
- Kubernetes node metricset
- Kubernetes pod metricset
- Kubernetes proxy metricset
- Kubernetes scheduler metricset
- Kubernetes state_container metricset
- Kubernetes state_cronjob metricset
- Kubernetes state_daemonset metricset
- Kubernetes state_deployment metricset
- Kubernetes state_job metricset
- Kubernetes state_node metricset
- Kubernetes state_persistentvolumeclaim metricset
- Kubernetes state_pod metricset
- Kubernetes state_replicaset metricset
- Kubernetes state_resourcequota metricset
- Kubernetes state_service metricset
- Kubernetes state_statefulset metricset
- Kubernetes state_storageclass metricset
- Kubernetes system metricset
- Kubernetes volume metricset
- KVM module
- Linux module
- Logstash module
- Memcached module
- MongoDB module
- MSSQL module
- Munin module
- MySQL module
- NATS module
- Nginx module
- Openmetrics module
- Oracle module
- PHP_FPM module
- PostgreSQL module
- Prometheus module
- RabbitMQ module
- Redis module
- Redis Enterprise module
- SQL module
- Stan module
- Statsd module
- SyncGateway module
- System module
- System core metricset
- System cpu metricset
- System diskio metricset
- System entropy metricset
- System filesystem metricset
- System fsstat metricset
- System load metricset
- System memory metricset
- System network metricset
- System network_summary metricset
- System process metricset
- System process_summary metricset
- System raid metricset
- System service metricset
- System socket metricset
- System socket_summary metricset
- System uptime metricset
- System users metricset
- Tomcat module
- Traefik module
- uWSGI module
- vSphere module
- Windows module
- ZooKeeper module
- Exported fields
- ActiveMQ fields
- Aerospike fields
- Airflow fields
- Apache fields
- AWS fields
- AWS Fargate fields
- Azure fields
- Beat fields
- Beat fields
- Ceph fields
- Cloud provider metadata fields
- Cloudfoundry fields
- CockroachDB fields
- Common fields
- Consul fields
- Containerd fields
- Coredns fields
- Couchbase fields
- CouchDB fields
- Docker fields
- Docker fields
- Dropwizard fields
- ECS fields
- Elasticsearch fields
- Enterprise Search fields
- Envoyproxy fields
- Etcd fields
- Google Cloud Platform fields
- Golang fields
- Graphite fields
- HAProxy fields
- Host fields
- HTTP fields
- IBM MQ fields
- IIS fields
- Istio fields
- Jolokia fields
- Jolokia Discovery autodiscover provider fields
- Kafka fields
- Kibana fields
- Kubernetes fields
- Kubernetes fields
- KVM fields
- Linux fields
- Logstash fields
- Memcached fields
- MongoDB fields
- MSSQL fields
- Munin fields
- MySQL fields
- NATS fields
- Nginx fields
- Openmetrics fields
- Oracle fields
- PHP_FPM fields
- PostgreSQL fields
- Process fields
- Prometheus fields
- Prometheus typed metrics fields
- RabbitMQ fields
- Redis fields
- Redis Enterprise fields
- SQL fields
- Stan fields
- Statsd fields
- SyncGateway fields
- System fields
- Tomcat fields
- Traefik fields
- uWSGI fields
- vSphere fields
- Windows fields
- ZooKeeper fields
- Monitor
- Secure
- Troubleshoot
- Get help
- Debug
- Understand logged metrics
- Common problems
- "open /compat/linux/proc: no such file or directory" error on FreeBSD
- Metricbeat collects system metrics for interfaces you didn’t configure
- Metricbeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Dashboard could not locate the index-pattern
- High RSS memory usage due to MADV settings
- Contribute to Beats
Azure module
editAzure module
editThe Azure Monitor feature collects and aggregates logs and metrics from a variety of sources into a common data platform where it can be used for analysis, visualization, and alerting.
The azure monitor metrics are numerical values that describe some aspect of a system at a particular point in time. They are collected at regular intervals and are identified with a timestamp, a name, a value, and one or more defining labels.
The azure module will periodically retrieve the azure monitor metrics using the Azure REST APIs as MetricList. Additional azure API calls will be executed in order to retrieve information regarding the resources targeted by the user.
Extra Azure charges on metric queries may be generated by this module. Please see additional notes about metrics and costs for more details.
Dashboards
editThe azure module comes with several predefined dashboards for virtual machines, VM guest metrics and virtual machine scale sets.
The VM overview dashboard shows information about CPU, memory, disk usage as well as operations per second. The two available filters help narrowing down the dashbord to specific regions and/or resource groups. For example:

If VM guest metrics are enabled then the guest metrics overview dashboard can help with monitoring ASP.NET applications and SQL Server metrics. For example:

The virtual machine scale sets dashboard is similar to the VM dashboard and shows relevant health information about running vm scale sets. For example:

The Azure storage dashboards show all relevant metrics for the blob, file, table and queue storage services:

The Azure billing dashboards show relevant usage and forecast information:

The Azure app_state dashboard shows relevant application insights information:

Module-specific configuration notes
editAll the tasks executed against the Azure Monitor REST API will use the Azure Resource Manager authentication model. Therefore, all requests must be authenticated with Azure Active Directory (Azure AD). One approach to authenticate the client application is to create an Azure AD service principal and retrieve the authentication (JWT) token. For a more detailed walk-through, have a look at using Azure PowerShell to create a service principal to access resources https://docs.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-2.7.0. It is also possible to create a service principal via the Azure portal https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal. Users will have to make sure the roles assigned to the application contain at least reading permissions to the monitor data, more on the roles here https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles.
Required credentials for the azure
module:
-
client_id
- The unique identifier for the application (also known as Application Id)
-
client_secret
- The client/application secret/key
-
subscription_id
- The unique identifier for the azure subscription
-
tenant_id
- The unique identifier of the Azure Active Directory instance
The azure credentials keys can be used if configured AZURE_CLIENT_ID
, AZURE_CLIENT_SECRET
, AZURE_TENANT_ID
, AZURE_SUBSCRIPTION_ID
-
resource_manager_endpoint
- string Optional, by default the azure public environment will be used, to override, users can provide a specific resource manager endpoint in order to use a different azure environment. Ex: https://management.chinacloudapi.cn for azure ChinaCloud https://management.microsoftazure.de for azure GermanCloud https://management.azure.com for azure PublicCloud https://management.usgovcloudapi.net for azure USGovernmentCloud
-
active_directory_endpoint
- string Optional, by default the associated active directory endpoint to the resource manager endpoint will be used, to override, users can provide a specific active directory endpoint in order to use a different azure environment. Ex: https://login.microsoftonline.com for azure ChinaCloud https://login.microsoftonline.us for azure GermanCloud https://login.chinacloudapi.cn for azure PublicCloud https://login.microsoftonline.de for azure USGovernmentCloud
-
resource_manager_audience
- string Optional, by default we are using the azure public environment, to override, users can provide a specific resource manager audience in order to use a different azure environment. Ex: https://management.chinacloudapi.cn/ for azure ChinaCloud https://management.microsoftazure.de/ for azure GermanCloud https://management.azure.com/ for azure PublicCloud https://management.usgovcloudapi.net/ for azure USGovernmentCloud Users can also use this in case of a Hybrid Cloud model, where one may define their own audiences.
Metricsets
editmonitor
editThis metricset allows users to retrieve metrics from specified resources. Added filters can apply here as the interval of retrieving these metrics, metric names,
aggregation list, namespaces and metric dimensions. The monitor metrics will have a minimum timegrain of 5 minutes, so the period
for monitor
metricset should be 300s
or multiples of 300s
.
compute_vm
editThis metricset will collect metrics from the virtual machines, these metrics will have a timegrain every 5 minutes,
so the period
for compute_vm
metricset should be 300s
or multiples of 300s
.
compute_vm_scaleset
editThis metricset will collect metrics from the virtual machine scalesets, these metrics will have a timegrain every 5 minutes,
so the period
for compute_vm_scaleset
metricset should be 300s
or multiples of 300s
.
storage
editThis metricset will collect metrics from the storage accounts, these metrics will have a timegrain every 5 minutes,
so the period
for storage
metricset should be 300s
or multiples of 300s
.
container_instance
editThis metricset will collect metrics from specified container groups, these metrics will have a timegrain every 5 minutes,
so the period
for container_instance
metricset should be 300s
or multiples of 300s
.
container_registry
editThis metricset will collect metrics from the container registries, these metrics will have a timegrain every 5 minutes,
so the period
for container_registry
metricset should be 300s
or multiples of 300s
.
container_service
editThis metricset will collect metrics from the container services, these metrics will have a timegrain every 5 minutes,
so the period
for container_service
metricset should be 300s
or multiples of 300s
.
database_account
editThis metricset will collect relevant metrics from specified database accounts, these metrics will have a timegrain every 5 minutes,
so the period
for database_account
metricset should be 300s
or multiples of 300s
.
billing
editThis metricset will collect relevant usage data and forecast information from a specific subscription, these metrics will have a timegrain every 24 hours,
so the period
for billing
metricset should be 24h
or multiples of 24h
.
app_insights
editThis metricset will collect application insights metrics, the period
(interval) for the app-insights
metricset is set by default at 300s
.
app_state
editThis metricset concentrate on the most relevant application insights metrics and provides a dashboard for visualization, the period
(interval) for the app_state
metricset is set by default at 300s
.
Additional notes about metrics and costs
editCosts: Metric queries are charged based on the number of standard API calls. More information on pricing here https://azure.microsoft.com/id-id/pricing/details/monitor/.
Authentication: we are handling authentication on our side (creating/renewing the authentication token), so we advise users to use dedicated credentials for metricbeat only.
The Azure module supports the standard configuration options that are described in Modules. Here is an example configuration:
metricbeat.modules: - module: azure metricsets: - monitor enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' resources: - resource_query: "resourceType eq 'Microsoft.DocumentDb/databaseAccounts'" metrics: - name: ["DataUsage", "DocumentCount", "DocumentQuota"] namespace: "Microsoft.DocumentDb/databaseAccounts" - module: azure metricsets: - compute_vm enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - compute_vm_scaleset enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - storage enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - container_instance enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - container_service enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - container_registry enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - database_account enabled: true period: 300s client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - billing enabled: true period: 24h client_id: '${AZURE_CLIENT_ID:""}' client_secret: '${AZURE_CLIENT_SECRET:""}' tenant_id: '${AZURE_TENANT_ID:""}' subscription_id: '${AZURE_SUBSCRIPTION_ID:""}' - module: azure metricsets: - app_insights enabled: true period: 300s application_id: '' api_key: '' metrics: - id: ["requests/count", "requests/duration"] - module: azure metricsets: - app_state enabled: true period: 300s application_id: '' api_key: ''
The following metricsets are available:
On this page