IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Google Cloud Platform loadbalancing metricset Google Cloud Platform storage metricset »
Elastic Docs ›Metricbeat Reference [7.7] ›Modules ›Google Cloud Platform module

Google Cloud Platform pubsub metricset

edit

Google Cloud Platform pubsub metricset

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

PubSub Metricset to fetch metrics from Pub/Sub topics and subscriptions in Google Cloud Platform.

The pubsub Metricset contains all GA stage metrics exported from the Stackdriver API. The field names have been left untouched for people already familiar with them.

No special permissions are needed apart from the ones detailed in the module section of the docs.

Snapshot Metrics

edit
  • pubsub.snapshot.backlog_bytes: Total byte size of the messages retained in a snapshot.
  • pubsub.snapshot.backlog_bytes_by_region: Total byte size of the messages retained in a snapshot, broken down by Cloud region.
  • pubsub.snapshot.config_updates_count: Cumulative count of configuration changes, grouped by operation type and result.
  • pubsub.snapshot.num_messages: Number of messages retained in a snapshot. Sampled every 60 seconds.
  • pubsub.snapshot.num_messages_by_region: Number of messages retained in a snapshot, broken down by Cloud region.
  • pubsub.snapshot.oldest_message_age: Age (in seconds) of the oldest message retained in a snapshot.
  • pubsub.snapshot.oldest_message_age_by_region: Age (in seconds) of the oldest message retained in a snapshot, broken down by Cloud region.

Subscription Metrics

edit
  • pubsub.subscription.ack_message_count: Cumulative count of messages acknowledged by Acknowledge requests, grouped by delivery type.
  • pubsub.subscription.backlog_bytes: Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription.
  • pubsub.subscription.byte_cost: Cumulative cost of operations, measured in bytes. This is used to measure quota utilization.
  • pubsub.subscription.config_updates_count: Cumulative count of configuration changes for each subscription, grouped by operation type and result.
  • pubsub.subscription.dead_letter_message_count: Cumulative count of messages published to dead letter topic, grouped by result.
  • pubsub.subscription.mod_ack_deadline_message_count: Cumulative count of messages whose deadline was updated by ModifyAckDeadline requests, grouped by delivery type.
  • pubsub.subscription.mod_ack_deadline_message_operation_count: Cumulative count of ModifyAckDeadline message operations, grouped by result.
  • pubsub.subscription.mod_ack_deadline_request_count: Cumulative count of ModifyAckDeadline requests, grouped by result.
  • pubsub.subscription.num_outstanding_messages: Number of messages delivered to a subscription’s push endpoint, but not yet acknowledged.
  • pubsub.subscription.num_undelivered_messages: Number of unacknowledged messages (a.k.a. backlog messages) in a subscription.
  • pubsub.subscription.oldest_retained_acked_message_age: Age (in seconds) of the oldest acknowledged message retained in a subscription.
  • pubsub.subscription.oldest_retained_acked_message_age_by_region: Age (in seconds) of the oldest acknowledged message retained in a subscription, broken down by Cloud region.
  • pubsub.subscription.oldest_unacked_message_age: Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription.
  • pubsub.subscription.oldest_unacked_message_age_by_region: Age (in seconds) of the oldest unacknowledged message in a subscription, broken down by Cloud region.
  • pubsub.subscription.pull_ack_message_operation_count: Cumulative count of acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
  • pubsub.subscription.pull_ack_request_count: Cumulative count of acknowledge requests, grouped by result.
  • pubsub.subscription.pull_message_operation_count: Cumulative count of pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
  • pubsub.subscription.pull_request_count: Cumulative count of pull requests, grouped by result.
  • pubsub.subscription.push_request_count: Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages. So each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times.
  • pubsub.subscription.push_request_latencies: Distribution of push request latencies (in microseconds), grouped by result.
  • pubsub.subscription.retained_acked_bytes: otal byte size of the acknowledged messages retained in a subscription.
  • pubsub.subscription.retained_acked_bytes_by_region: Total byte size of the acknowledged messages retained in a subscription, broken down by Cloud region.
  • pubsub.subscription.seek_request_count: Cumulative count of seek attempts, grouped by result.
  • pubsub.subscription.sent_message_count: Cumulative count of messages sent by Cloud Pub/Sub to subscriber clients, grouped by delivery type.
  • pubsub.subscription.streaming_pull_ack_message_operation_count: Cumulative count of StreamingPull acknowledge message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
  • pubsub.subscription.streaming_pull_ack_request_count: Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result.
  • pubsub.subscription.streaming_pull_message_operation_count: Cumulative count of streaming pull message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric <code>subscription/mod_ack_deadline_message_operation_count
  • pubsub.subscription.streaming_pull_mod_ack_deadline_message_operation_count: Cumulative count of StreamingPull ModifyAckDeadline operations, grouped by result.
  • pubsub.subscription.streaming_pull_mod_ack_deadline_request_count: Cumulative count of streaming pull requests with non-empty ModifyAckDeadline fields, grouped by result.
  • pubsub.subscription.streaming_pull_response_count: Cumulative count of streaming pull responses, grouped by result.
  • pubsub.subscription.unacked_bytes_by_region: Total byte size of the unacknowledged messages in a subscription, broken down by Cloud region.

Topic Metrics

edit
  • pubsub.topic.byte_cost: Cost of operations, measured in bytes. This is used to measure utilization for quotas.
  • pubsub.topic.config_updates_count: Cumulative count of configuration changes, grouped by operation type and result.
  • pubsub.topic.message_sizes: Distribution of publish message sizes (in bytes).
  • pubsub.topic.oldest_retained_acked_message_age_by_region: Age (in seconds) of the oldest acknowledged message retained in a topic, broken down by Cloud region.
  • pubsub.topic.oldest_unacked_message_age_by_region: Age (in seconds) of the oldest unacknowledged message in a topic, broken down by Cloud region.
  • pubsub.topic.retained_acked_bytes_by_region: Total byte size of the acknowledged messages retained in a topic, broken down by Cloud region.
  • pubsub.topic.send_message_operation_count: Cumulative count of publish message operations, grouped by result. For a definition of message operations, see Cloud Pub/Sub metric subscription/mod_ack_deadline_message_operation_count.
  • pubsub.topic.send_request_count: Cumulative count of publish requests, grouped by result.
  • pubsub.topic.unacked_bytes_by_region: Total byte size of the unacknowledged messages in a topic, broken down by Cloud region.

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

Fields

edit

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "cloud": {
        "account": {
            "id": "elastic-observability"
        },
        "provider": "googlecloud"
    },
    "event": {
        "dataset": "googlecloud.pubsub",
        "duration": 115000,
        "module": "googlecloud"
    },
    "googlecloud": {
        "labels": {
            "resource": {
                "subscription_id": "test-ks"
            }
        },
        "pubsub": {
            "snapshot": {
                "backlog_bytes": 19,
                "backlog_bytes_by_region": 19,
                "num_messages": 4,
                "num_messages_by_region": 4,
                "oldest_message_age": 69319,
                "oldest_message_age_by_region": 69319
            },
            "subscription": {
                "backlog_bytes": 0,
                "num_undelivered_messages": 0,
                "oldest_retained_acked_message_age": 0,
                "oldest_retained_acked_message_age_by_region": 0,
                "oldest_unacked_message_age": 0,
                "oldest_unacked_message_age_by_region": 69277,
                "retained_acked_bytes": 0,
                "retained_acked_bytes_by_region": 0,
                "unacked_bytes_by_region": 19
            },
            "topic": {
                "oldest_retained_acked_message_age_by_region": 0,
                "oldest_unacked_message_age_by_region": 69319,
                "retained_acked_bytes_by_region": 0,
                "unacked_bytes_by_region": 76
            }
        }
    },
    "metricset": {
        "name": "pubsub",
        "period": 10000
    },
    "service": {
        "type": "googlecloud"
    }
}
« Google Cloud Platform loadbalancing metricset Google Cloud Platform storage metricset »