IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Circuit breaker errors High JVM memory pressure »
Elastic Docs ›Elasticsearch Guide [8.7] ›Troubleshooting ›Fix common cluster issues

High CPU usage

edit

High CPU usage

edit

Elasticsearch uses thread pools to manage CPU resources for concurrent operations. High CPU usage typically means one or more thread pools are running low.

If a thread pool is depleted, Elasticsearch will reject requests related to the thread pool. For example, if the search thread pool is depleted, Elasticsearch will reject search requests until more threads are available.

Diagnose high CPU usage

edit

Check CPU usage

From your deployment menu, click Performance. The page’s CPU Usage chart shows your deployment’s CPU usage as a percentage.

High CPU usage can also deplete your CPU credits. CPU credits let Elasticsearch Service provide smaller clusters with a performance boost when needed. The CPU credits chart shows your remaining CPU credits, measured in seconds of CPU time.

You can also use the cat nodes API to get the current CPU usage for each node.

response = client.cat.nodes(
  v: true,
  s: 'cpu:desc'
)
puts response
GET _cat/nodes?v=true&s=cpu:desc

The response’s cpu column contains the current CPU usage as a percentage. The node column contains the node’s name.

Check hot threads

If a node has high CPU usage, use the nodes hot threads API to check for resource-intensive threads running on the node.

response = client.nodes.hot_threads(
  node_id: 'my-node,my-other-node'
)
puts response
GET _nodes/my-node,my-other-node/hot_threads

This API returns a breakdown of any hot threads in plain text.

Reduce CPU usage

edit

The following tips outline the most common causes of high CPU usage and their solutions.

Scale your cluster

Heavy indexing and search loads can deplete smaller thread pools. To better handle heavy workloads, add more nodes to your cluster or upgrade your existing nodes to increase capacity.

Spread out bulk requests

While more efficient than individual requests, large bulk indexing or multi-search requests still require CPU resources. If possible, submit smaller requests and allow more time between them.

Cancel long-running searches

Long-running searches can block threads in the search thread pool. To check for these searches, use the task management API.

response = client.tasks.list(
  actions: '*search',
  detailed: true
)
puts response
GET _tasks?actions=*search&detailed

The response’s description contains the search request and its queries. running_time_in_nanos shows how long the search has been running.

{
  "nodes" : {
    "oTUltX4IQMOUUVeiohTt8A" : {
      "name" : "my-node",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1:9300",
      "tasks" : {
        "oTUltX4IQMOUUVeiohTt8A:464" : {
          "node" : "oTUltX4IQMOUUVeiohTt8A",
          "id" : 464,
          "type" : "transport",
          "action" : "indices:data/read/search",
          "description" : "indices[my-index], search_type[QUERY_THEN_FETCH], source[{\"query\":...}]",
          "start_time_in_millis" : 4081771730000,
          "running_time_in_nanos" : 13991383,
          "cancellable" : true
        }
      }
    }
  }
}

To cancel a search and free up resources, use the API’s _cancel endpoint.

response = client.tasks.cancel(
  task_id: 'oTUltX4IQMOUUVeiohTt8A:464'
)
puts response
POST _tasks/oTUltX4IQMOUUVeiohTt8A:464/_cancel

For additional tips on how to track and avoid resource-intensive searches, see Avoid expensive searches.

« Circuit breaker errors High JVM memory pressure »