Get component templates Added in 5.1.0
Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
Path parameters
-
The name of the component template. It accepts wildcard expressions. If it is omitted, all component templates are returned.
Query parameters
-
local boolean
If
true
, the request computes the list of selected nodes from the local cluster state. Iffalse
the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node. -
master_timeout string
The period to wait for a connection to the master node.
curl \
--request GET http://api.example.com/_cat/component_templates/{name}
[
{
"name": "string",
"version": "string",
"alias_count": "string",
"mapping_count": "string",
"settings_count": "string",
"metadata_count": "string",
"included_in": "string"
}
]
alias index filter routing.index routing.search is_write_index
alias1 test1 - - - -
alias2 test1 * - - -
alias3 test1 - 1 1 -
alias4 test1 - 2 1,2 -
Get index template information Added in 5.2.0
Get information about the index templates in a cluster. You can use index templates to apply index settings and field mappings to new indices at creation. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get index template API.
Path parameters
-
The name of the template to return. Accepts wildcard expressions. If omitted, all templates are returned.
Query parameters
-
local boolean
If
true
, the request computes the list of selected nodes from the local cluster state. Iffalse
the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node. -
master_timeout string
Period to wait for a connection to the master node.
curl \
--request GET http://api.example.com/_cat/templates/{name}
[
{
"name": "string",
"index_patterns": "string",
"order": "string",
"version": "string",
"composed_of": "string"
}
]
name index_patterns order version composed_of
my-template-0 [te*] 500 []
my-template-1 [tea*] 501 []
my-template-2 [teak*] 502 7 []
Reroute the cluster Added in 5.0.0
Manually change the allocation of individual shards in the cluster. For example, a shard can be moved from one node to another explicitly, an allocation can be canceled, and an unassigned shard can be explicitly allocated to a specific node.
It is important to note that after processing any reroute commands Elasticsearch will perform rebalancing as normal (respecting the values of settings such as cluster.routing.rebalance.enable
) in order to remain in a balanced state.
For example, if the requested allocation includes moving a shard from node1 to node2 then this may cause a shard to be moved from node2 back to node1 to even things out.
The cluster can be set to disable allocations using the cluster.routing.allocation.enable
setting.
If allocations are disabled then the only allocations that will be performed are explicit ones given using the reroute command, and consequent allocations due to rebalancing.
The cluster will attempt to allocate a shard a maximum of index.allocation.max_retries
times in a row (defaults to 5
), before giving up and leaving the shard unallocated.
This scenario can be caused by structural problems such as having an analyzer which refers to a stopwords file which doesn’t exist on all nodes.
Once the problem has been corrected, allocation can be manually retried by calling the reroute API with the ?retry_failed
URI query parameter, which will attempt a single retry round for these shards.
Query parameters
-
dry_run boolean
If true, then the request simulates the operation. It will calculate the result of applying the commands to the current cluster state and return the resulting cluster state after the commands (and rebalancing) have been applied; it will not actually perform the requested changes.
-
explain boolean
If true, then the response contains an explanation of why the commands can or cannot run.
-
metric string | array[string]
Limits the information returned to the specified metrics.
-
retry_failed boolean
If true, then retries allocation of shards that are blocked due to too many subsequent allocation failures.
-
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST http://api.example.com/_cluster/reroute \
--header "Content-Type: application/json" \
--data '"{\n \"commands\": [\n {\n \"move\": {\n \"index\": \"test\", \"shard\": 0,\n \"from_node\": \"node1\", \"to_node\": \"node2\"\n }\n },\n {\n \"allocate_replica\": {\n \"index\": \"test\", \"shard\": 1,\n \"node\": \"node3\"\n }\n }\n ]\n}"'
{
"commands": [
{
"move": {
"index": "test", "shard": 0,
"from_node": "node1", "to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test", "shard": 1,
"node": "node3"
}
}
]
}
{
"acknowledged": true,
"explanations": [
{
"command": "string",
"decisions": [
{
"decider": "string",
"decision": "string",
"explanation": "string"
}
],
"parameters": {
"allow_primary": true,
"index": "string",
"node": "string",
"shard": 42.0,
"from_node": "string",
"to_node": "string"
}
}
],
"state": {}
}
Get node statistics
Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.
Path parameters
-
Comma-separated list of node IDs or names used to limit returned information.
-
Limit the information returned to the specified metrics
-
Limit the information returned for indices metric to the specific index metrics. It can be used only if indices (or all) metric is specified.
Query parameters
-
completion_fields string | array[string]
Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.
-
fielddata_fields string | array[string]
Comma-separated list or wildcard expressions of fields to include in fielddata statistics.
-
fields string | array[string]
Comma-separated list or wildcard expressions of fields to include in the statistics.
-
groups boolean
Comma-separated list of search groups to include in the search statistics.
-
include_segment_file_sizes boolean
If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).
-
level string
Indicates whether statistics are aggregated at the cluster, index, or shard level.
Values are
cluster
,indices
, orshards
. -
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
-
types array[string]
A comma-separated list of document types for the indexing index metric.
-
include_unloaded_segments boolean
If
true
, the response includes information from segments that are not loaded into memory.
curl \
--request GET http://api.example.com/_nodes/{node_id}/stats/{metric}/{index_metric}
{
"_nodes": {
"failures": [
{
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
],
"total": 42.0,
"successful": 42.0,
"failed": 42.0
},
"cluster_name": "string",
"nodes": {
"additionalProperty1": {
"adaptive_selection": {
"additionalProperty1": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
},
"additionalProperty2": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
}
},
"breakers": {
"additionalProperty1": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
},
"additionalProperty2": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
}
},
"fs": {
"data": [
{}
],
"timestamp": 42.0,
"total": {
"available": "string",
"available_in_bytes": 42.0,
"free": "string",
"free_in_bytes": 42.0,
"total": "string",
"total_in_bytes": 42.0
},
"io_stats": {
"devices": [
{}
],
"total": {}
}
},
"host": "string",
"http": {
"current_open": 42.0,
"total_opened": 42.0,
"clients": [
{}
],
"routes": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"ingest": {
"pipelines": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"total": {
"count": 42.0,
"current": 42.0,
"failed": 42.0
}
},
"ip": "string",
"jvm": {
"buffer_pools": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"classes": {
"current_loaded_count": 42.0,
"total_loaded_count": 42.0,
"total_unloaded_count": 42.0
},
"gc": {
"collectors": {}
},
"mem": {
"heap_used_in_bytes": 42.0,
"heap_used_percent": 42.0,
"heap_committed_in_bytes": 42.0,
"heap_max_in_bytes": 42.0,
"non_heap_used_in_bytes": 42.0,
"non_heap_committed_in_bytes": 42.0,
"pools": {}
},
"threads": {
"count": 42.0,
"peak_count": 42.0
},
"timestamp": 42.0,
"uptime": "string",
"uptime_in_millis": 42.0
},
"name": "string",
"os": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"": {},
"swap": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"cgroup": {
"cpuacct": {},
"cpu": {},
"memory": {}
},
"timestamp": 42.0
},
"process": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"mem": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"open_file_descriptors": 42.0,
"max_file_descriptors": 42.0,
"timestamp": 42.0
},
"roles": [
"master"
],
"script": {
"cache_evictions": 42.0,
"compilations": 42.0,
"compilations_history": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"compilation_limit_triggered": 42.0,
"contexts": [
{}
]
},
"script_cache": {},
"thread_pool": {
"additionalProperty1": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
},
"additionalProperty2": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
}
},
"timestamp": 42.0,
"transport": {
"inbound_handling_time_histogram": [
{}
],
"outbound_handling_time_histogram": [
{}
],
"rx_count": 42.0,
"rx_size": "string",
"rx_size_in_bytes": 42.0,
"server_open": 42.0,
"tx_count": 42.0,
"tx_size": "string",
"tx_size_in_bytes": 42.0,
"total_outbound_connections": 42.0
},
"transport_address": "string",
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"discovery": {
"cluster_state_queue": {
"total": 42.0,
"pending": 42.0,
"committed": 42.0
},
"published_cluster_states": {
"full_states": 42.0,
"incompatible_diffs": 42.0,
"compatible_diffs": 42.0
},
"cluster_state_update": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"serialized_cluster_states": {
"full_states": {},
"diffs": {}
},
"cluster_applier_stats": {
"recordings": [
{}
]
}
},
"indexing_pressure": {
"memory": {
"limit_in_bytes": 42.0,
"current": {},
"total": {}
}
},
"indices": {
"commit": {
"generation": 42.0,
"id": "string",
"num_docs": 42.0,
"user_data": {}
},
"completion": {
"size_in_bytes": 42.0,
"fields": {}
},
"docs": {
"count": 42.0,
"deleted": 42.0
},
"fielddata": {
"evictions": 42.0,
"memory_size_in_bytes": 42.0,
"fields": {}
},
"flush": {
"periodic": 42.0,
"total": 42.0,
"total_time": "string"
},
"get": {
"current": 42.0,
"exists_time": "string",
"exists_total": 42.0,
"missing_time": "string",
"missing_total": 42.0,
"time": "string",
"total": 42.0
},
"indexing": {
"index_current": 42.0,
"delete_current": 42.0,
"delete_time": "string",
"delete_total": 42.0,
"is_throttled": true,
"noop_update_total": 42.0,
"throttle_time": "string",
"index_time": "string",
"index_total": 42.0,
"index_failed": 42.0,
"types": {},
"write_load": 42.0
},
"mappings": {
"total_count": 42.0,
"total_estimated_overhead_in_bytes": 42.0
},
"merges": {
"current": 42.0,
"current_docs": 42.0,
"current_size": "string",
"current_size_in_bytes": 42.0,
"total": 42.0,
"total_auto_throttle": "string",
"total_auto_throttle_in_bytes": 42.0,
"total_docs": 42.0,
"total_size": "string",
"total_size_in_bytes": 42.0,
"total_stopped_time": "string",
"total_throttled_time": "string",
"total_time": "string"
},
"shard_path": {
"data_path": "string",
"is_custom_data_path": true,
"state_path": "string"
},
"query_cache": {
"cache_count": 42.0,
"cache_size": 42.0,
"evictions": 42.0,
"hit_count": 42.0,
"memory_size_in_bytes": 42.0,
"miss_count": 42.0,
"total_count": 42.0
},
"recovery": {
"current_as_source": 42.0,
"current_as_target": 42.0,
"throttle_time": "string"
},
"refresh": {
"external_total": 42.0,
"listeners": 42.0,
"total": 42.0,
"total_time": "string"
},
"request_cache": {
"evictions": 42.0,
"hit_count": 42.0,
"memory_size": "string",
"memory_size_in_bytes": 42.0,
"miss_count": 42.0
},
"retention_leases": {
"primary_term": 42.0,
"version": 42.0,
"leases": [
{}
]
},
"routing": {
"node": "string",
"primary": true,
"state": "UNASSIGNED"
},
"search": {
"fetch_current": 42.0,
"fetch_time": "string",
"fetch_total": 42.0,
"open_contexts": 42.0,
"query_current": 42.0,
"query_time": "string",
"query_total": 42.0,
"scroll_current": 42.0,
"scroll_time": "string",
"scroll_total": 42.0,
"suggest_current": 42.0,
"suggest_time": "string",
"suggest_total": 42.0,
"groups": {}
},
"segments": {
"count": 42.0,
"doc_values_memory_in_bytes": 42.0,
"file_sizes": {},
"fixed_bit_set_memory_in_bytes": 42.0,
"index_writer_max_memory_in_bytes": 42.0,
"index_writer_memory_in_bytes": 42.0,
"max_unsafe_auto_id_timestamp": 42.0,
"memory_in_bytes": 42.0,
"norms_memory_in_bytes": 42.0,
"points_memory_in_bytes": 42.0,
"stored_fields_memory_in_bytes": 42.0,
"terms_memory_in_bytes": 42.0,
"term_vectors_memory_in_bytes": 42.0,
"version_map_memory_in_bytes": 42.0
},
"seq_no": {
"global_checkpoint": 42.0,
"local_checkpoint": 42.0,
"max_seq_no": 42.0
},
"store": {
"size_in_bytes": 42.0,
"reserved_in_bytes": 42.0,
"total_data_set_size_in_bytes": 42.0
},
"translog": {
"earliest_last_modified_age": 42.0,
"operations": 42.0,
"size": "string",
"size_in_bytes": 42.0,
"uncommitted_operations": 42.0,
"uncommitted_size": "string",
"uncommitted_size_in_bytes": 42.0
},
"warmer": {
"current": 42.0,
"total": 42.0,
"total_time": "string"
},
"bulk": {
"total_operations": 42.0,
"total_time": "string",
"total_size_in_bytes": 42.0,
"avg_time": "string",
"avg_size_in_bytes": 42.0
},
"shards": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"shard_stats": {
"total_count": 42.0
},
"additionalProperty1": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
},
"additionalProperty2": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
}
}
},
"additionalProperty2": {
"adaptive_selection": {
"additionalProperty1": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
},
"additionalProperty2": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
}
},
"breakers": {
"additionalProperty1": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
},
"additionalProperty2": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
}
},
"fs": {
"data": [
{}
],
"timestamp": 42.0,
"total": {
"available": "string",
"available_in_bytes": 42.0,
"free": "string",
"free_in_bytes": 42.0,
"total": "string",
"total_in_bytes": 42.0
},
"io_stats": {
"devices": [
{}
],
"total": {}
}
},
"host": "string",
"http": {
"current_open": 42.0,
"total_opened": 42.0,
"clients": [
{}
],
"routes": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"ingest": {
"pipelines": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"total": {
"count": 42.0,
"current": 42.0,
"failed": 42.0
}
},
"ip": "string",
"jvm": {
"buffer_pools": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"classes": {
"current_loaded_count": 42.0,
"total_loaded_count": 42.0,
"total_unloaded_count": 42.0
},
"gc": {
"collectors": {}
},
"mem": {
"heap_used_in_bytes": 42.0,
"heap_used_percent": 42.0,
"heap_committed_in_bytes": 42.0,
"heap_max_in_bytes": 42.0,
"non_heap_used_in_bytes": 42.0,
"non_heap_committed_in_bytes": 42.0,
"pools": {}
},
"threads": {
"count": 42.0,
"peak_count": 42.0
},
"timestamp": 42.0,
"uptime": "string",
"uptime_in_millis": 42.0
},
"name": "string",
"os": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"": {},
"swap": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"cgroup": {
"cpuacct": {},
"cpu": {},
"memory": {}
},
"timestamp": 42.0
},
"process": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"mem": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"open_file_descriptors": 42.0,
"max_file_descriptors": 42.0,
"timestamp": 42.0
},
"roles": [
"master"
],
"script": {
"cache_evictions": 42.0,
"compilations": 42.0,
"compilations_history": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"compilation_limit_triggered": 42.0,
"contexts": [
{}
]
},
"script_cache": {},
"thread_pool": {
"additionalProperty1": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
},
"additionalProperty2": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
}
},
"timestamp": 42.0,
"transport": {
"inbound_handling_time_histogram": [
{}
],
"outbound_handling_time_histogram": [
{}
],
"rx_count": 42.0,
"rx_size": "string",
"rx_size_in_bytes": 42.0,
"server_open": 42.0,
"tx_count": 42.0,
"tx_size": "string",
"tx_size_in_bytes": 42.0,
"total_outbound_connections": 42.0
},
"transport_address": "string",
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"discovery": {
"cluster_state_queue": {
"total": 42.0,
"pending": 42.0,
"committed": 42.0
},
"published_cluster_states": {
"full_states": 42.0,
"incompatible_diffs": 42.0,
"compatible_diffs": 42.0
},
"cluster_state_update": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"serialized_cluster_states": {
"full_states": {},
"diffs": {}
},
"cluster_applier_stats": {
"recordings": [
{}
]
}
},
"indexing_pressure": {
"memory": {
"limit_in_bytes": 42.0,
"current": {},
"total": {}
}
},
"indices": {
"commit": {
"generation": 42.0,
"id": "string",
"num_docs": 42.0,
"user_data": {}
},
"completion": {
"size_in_bytes": 42.0,
"fields": {}
},
"docs": {
"count": 42.0,
"deleted": 42.0
},
"fielddata": {
"evictions": 42.0,
"memory_size_in_bytes": 42.0,
"fields": {}
},
"flush": {
"periodic": 42.0,
"total": 42.0,
"total_time": "string"
},
"get": {
"current": 42.0,
"exists_time": "string",
"exists_total": 42.0,
"missing_time": "string",
"missing_total": 42.0,
"time": "string",
"total": 42.0
},
"indexing": {
"index_current": 42.0,
"delete_current": 42.0,
"delete_time": "string",
"delete_total": 42.0,
"is_throttled": true,
"noop_update_total": 42.0,
"throttle_time": "string",
"index_time": "string",
"index_total": 42.0,
"index_failed": 42.0,
"types": {},
"write_load": 42.0
},
"mappings": {
"total_count": 42.0,
"total_estimated_overhead_in_bytes": 42.0
},
"merges": {
"current": 42.0,
"current_docs": 42.0,
"current_size": "string",
"current_size_in_bytes": 42.0,
"total": 42.0,
"total_auto_throttle": "string",
"total_auto_throttle_in_bytes": 42.0,
"total_docs": 42.0,
"total_size": "string",
"total_size_in_bytes": 42.0,
"total_stopped_time": "string",
"total_throttled_time": "string",
"total_time": "string"
},
"shard_path": {
"data_path": "string",
"is_custom_data_path": true,
"state_path": "string"
},
"query_cache": {
"cache_count": 42.0,
"cache_size": 42.0,
"evictions": 42.0,
"hit_count": 42.0,
"memory_size_in_bytes": 42.0,
"miss_count": 42.0,
"total_count": 42.0
},
"recovery": {
"current_as_source": 42.0,
"current_as_target": 42.0,
"throttle_time": "string"
},
"refresh": {
"external_total": 42.0,
"listeners": 42.0,
"total": 42.0,
"total_time": "string"
},
"request_cache": {
"evictions": 42.0,
"hit_count": 42.0,
"memory_size": "string",
"memory_size_in_bytes": 42.0,
"miss_count": 42.0
},
"retention_leases": {
"primary_term": 42.0,
"version": 42.0,
"leases": [
{}
]
},
"routing": {
"node": "string",
"primary": true,
"state": "UNASSIGNED"
},
"search": {
"fetch_current": 42.0,
"fetch_time": "string",
"fetch_total": 42.0,
"open_contexts": 42.0,
"query_current": 42.0,
"query_time": "string",
"query_total": 42.0,
"scroll_current": 42.0,
"scroll_time": "string",
"scroll_total": 42.0,
"suggest_current": 42.0,
"suggest_time": "string",
"suggest_total": 42.0,
"groups": {}
},
"segments": {
"count": 42.0,
"doc_values_memory_in_bytes": 42.0,
"file_sizes": {},
"fixed_bit_set_memory_in_bytes": 42.0,
"index_writer_max_memory_in_bytes": 42.0,
"index_writer_memory_in_bytes": 42.0,
"max_unsafe_auto_id_timestamp": 42.0,
"memory_in_bytes": 42.0,
"norms_memory_in_bytes": 42.0,
"points_memory_in_bytes": 42.0,
"stored_fields_memory_in_bytes": 42.0,
"terms_memory_in_bytes": 42.0,
"term_vectors_memory_in_bytes": 42.0,
"version_map_memory_in_bytes": 42.0
},
"seq_no": {
"global_checkpoint": 42.0,
"local_checkpoint": 42.0,
"max_seq_no": 42.0
},
"store": {
"size_in_bytes": 42.0,
"reserved_in_bytes": 42.0,
"total_data_set_size_in_bytes": 42.0
},
"translog": {
"earliest_last_modified_age": 42.0,
"operations": 42.0,
"size": "string",
"size_in_bytes": 42.0,
"uncommitted_operations": 42.0,
"uncommitted_size": "string",
"uncommitted_size_in_bytes": 42.0
},
"warmer": {
"current": 42.0,
"total": 42.0,
"total_time": "string"
},
"bulk": {
"total_operations": 42.0,
"total_time": "string",
"total_size_in_bytes": 42.0,
"avg_time": "string",
"avg_size_in_bytes": 42.0
},
"shards": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"shard_stats": {
"total_count": 42.0
},
"additionalProperty1": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
},
"additionalProperty2": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
}
}
}
}
}
Update the connector error field Technical preview
Set the error field for the connector. If the error provided in the request body is non-null, the connector’s status is updated to error. Otherwise, if the error is reset to null, the connector status is updated to connected.
Path parameters
-
The unique identifier of the connector to be updated
curl \
--request PUT http://api.example.com/_connector/{connector_id}/_error \
--header "Content-Type: application/json" \
--data '"{\n \"error\": \"Houston, we have a problem!\"\n}"'
{
"error": "Houston, we have a problem!"
}
{
"result": "updated"
}
Get term vector information
Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields
parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime
parameter to false
.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
- term frequency in the field (always returned)
- term positions (
positions: true
) - start and end offsets (
offsets: true
) - term payloads (
payloads: true
), as base64 encoded bytes
If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing
only to hit a particular shard.
Path parameters
-
The name of the index that contains the document.
Query parameters
-
fields string | array[string]
A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the
completion_fields
orfielddata_fields
parameters. -
field_statistics boolean
If
true
, the response includes:- The document count (how many documents contain this field).
- The sum of document frequencies (the sum of document frequencies for all terms in this field).
- The sum of total term frequencies (the sum of total term frequencies of each term in this field).
-
offsets boolean
If
true
, the response includes term offsets. -
payloads boolean
If
true
, the response includes term payloads. -
positions boolean
If
true
, the response includes term positions. -
preference string
The node or shard the operation should be performed on. It is random by default.
-
realtime boolean
If true, the request is real-time as opposed to near-real-time.
-
routing string
A custom value that is used to route operations to a specific shard.
-
term_statistics boolean
If
true
, the response includes:- The total term frequency (how often a term occurs in all documents).
- The document frequency (the number of documents containing the current term).
By default these values are not returned since term statistics can have a serious performance impact.
-
version number
If
true
, returns the document version as part of a hit. -
version_type string
The version type.
Values are
internal
,external
,external_gte
, orforce
.
Body
-
doc object
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Additional properties are allowed.
-
filter object
Additional properties are allowed.
-
per_field_analyzer object
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
curl \
--request POST http://api.example.com/{index}/_termvectors \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'
{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}
{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}
{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}
{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}
{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}
{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}
{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}
{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}
Get index templates Added in 7.9.0
Get information about one or more index templates.
Query parameters
-
local boolean
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
-
flat_settings boolean
If true, returns settings in flat format.
-
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
include_defaults boolean
If true, returns all relevant default configurations for the index template.
curl \
--request GET http://api.example.com/_index_template
{
"index_templates": [
{
"name": "string",
"index_template": {
"index_patterns": "string",
"composed_of": [
"string"
],
"template": {
"aliases": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"mappings": {
"all_field": {},
"date_detection": true,
"dynamic": "strict",
"dynamic_date_formats": [
"string"
],
"dynamic_templates": [
{}
],
"_field_names": {},
"index_field": {},
"_meta": {},
"numeric_detection": true,
"properties": {},
"_routing": {},
"_size": {},
"_source": {},
"runtime": {},
"enabled": true,
"subobjects": true,
"_data_stream_timestamp": {}
},
"settings": {
"index": {},
"mode": "string",
"soft_deletes": {},
"sort": {},
"number_of_routing_shards": 42.0,
"check_on_startup": "true",
"codec": "string",
"load_fixed_bitset_filters_eagerly": true,
"auto_expand_replicas": "string",
"merge": {},
"search": {},
"refresh_interval": "string",
"max_result_window": 42.0,
"max_inner_result_window": 42.0,
"max_rescore_window": 42.0,
"max_docvalue_fields_search": 42.0,
"max_script_fields": 42.0,
"max_ngram_diff": 42.0,
"max_shingle_diff": 42.0,
"blocks": {},
"max_refresh_listeners": 42.0,
"analyze": {},
"highlight": {},
"max_terms_count": 42.0,
"max_regex_length": 42.0,
"routing": {},
"gc_deletes": "string",
"default_pipeline": "string",
"final_pipeline": "string",
"lifecycle": {},
"provided_name": "string",
"uuid": "string",
"version": {},
"max_slices_per_scroll": 42.0,
"translog": {},
"query_string": {},
"top_metrics_max_size": 42.0,
"analysis": {},
"settings": {},
"time_series": {},
"queries": {},
"similarity": {},
"mapping": {},
"indexing.slowlog": {},
"indexing_pressure": {},
"store": {}
},
"": {}
},
"version": 42.0,
"priority": 42.0,
"_meta": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"allow_auto_create": true,
"data_stream": {
"hidden": true,
"allow_custom_routing": true
},
"deprecated": true,
"ignore_missing_component_templates": "string"
}
}
]
}
Get index settings
Get setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.
Path parameters
-
Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (
*
). To target all data streams and indices, omit this parameter or use*
or_all
.
Query parameters
-
allow_no_indices boolean
If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*
returns an error if an index starts with foo but no index starts withbar
. -
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. -
flat_settings boolean
If
true
, returns settings in flat format. -
include_defaults boolean
If
true
, return all default settings in the response. -
local boolean
If
true
, the request retrieves information from the local node only. Iffalse
, information is retrieved from the master node. -
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET http://api.example.com/{index}/_settings
{
"*": {
"aliases": {
"additionalProperty1": {
"filter": {},
"index_routing": "string",
"is_hidden": true,
"is_write_index": true,
"routing": "string",
"search_routing": "string"
},
"additionalProperty2": {
"filter": {},
"index_routing": "string",
"is_hidden": true,
"is_write_index": true,
"routing": "string",
"search_routing": "string"
}
},
"mappings": {
"all_field": {
"analyzer": "string",
"enabled": true,
"omit_norms": true,
"search_analyzer": "string",
"similarity": "string",
"store": true,
"store_term_vector_offsets": true,
"store_term_vector_payloads": true,
"store_term_vector_positions": true,
"store_term_vectors": true
},
"date_detection": true,
"dynamic": "strict",
"dynamic_date_formats": [
"string"
],
"dynamic_templates": [
{}
],
"_field_names": {
"enabled": true
},
"index_field": {
"enabled": true
},
"_meta": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"numeric_detection": true,
"properties": {},
"_routing": {
"required": true
},
"_size": {
"enabled": true
},
"_source": {
"compress": true,
"compress_threshold": "string",
"enabled": true,
"excludes": [
"string"
],
"includes": [
"string"
],
"mode": "disabled"
},
"runtime": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {},
"options": {}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {},
"options": {}
},
"type": "boolean"
}
},
"enabled": true,
"subobjects": true,
"_data_stream_timestamp": {
"enabled": true
}
},
"settings": {
"index": {},
"mode": "string",
"routing_path": "string",
"soft_deletes": {
"enabled": true,
"retention_lease": {
"period": "string"
}
},
"sort": {
"field": "string",
"order": "asc",
"mode": "min",
"missing": "_last"
},
"number_of_shards": 42.0,
"number_of_replicas": 42.0,
"number_of_routing_shards": 42.0,
"check_on_startup": "true",
"codec": "string",
"": "string",
"load_fixed_bitset_filters_eagerly": true,
"hidden": true,
"auto_expand_replicas": "string",
"merge": {
"scheduler": {
"": 42.0
}
},
"search": {
"idle": {
"after": "string"
},
"slowlog": {
"level": "string",
"source": 42.0,
"reformat": true,
"threshold": {
"query": {},
"fetch": {}
}
}
},
"refresh_interval": "string",
"max_result_window": 42.0,
"max_inner_result_window": 42.0,
"max_rescore_window": 42.0,
"max_docvalue_fields_search": 42.0,
"max_script_fields": 42.0,
"max_ngram_diff": 42.0,
"max_shingle_diff": 42.0,
"blocks": {
"": true
},
"max_refresh_listeners": 42.0,
"analyze": {
"": 42.0
},
"highlight": {
"max_analyzed_offset": 42.0
},
"max_terms_count": 42.0,
"max_regex_length": 42.0,
"routing": {
"allocation": {
"enable": "all",
"include": {
"_tier_preference": "string",
"_id": "string"
},
"initial_recovery": {
"_id": "string"
},
"disk": {}
},
"rebalance": {
"enable": "all"
}
},
"gc_deletes": "string",
"default_pipeline": "string",
"final_pipeline": "string",
"lifecycle": {
"name": "string",
"": true,
"origination_date": 42.0,
"parse_origination_date": true,
"step": {
"wait_time_threshold": "string"
},
"rollover_alias": "string",
"prefer_ilm": true
},
"provided_name": "string",
"uuid": "string",
"version": {
"created": "string",
"created_string": "string"
},
"verified_before_close": true,
"format": "string",
"max_slices_per_scroll": 42.0,
"translog": {
"sync_interval": "string",
"durability": "request",
"": 42.0,
"retention": {
"": 42.0,
"age": "string"
}
},
"query_string": {
"": true
},
"priority": 42.0,
"top_metrics_max_size": 42.0,
"analysis": {
"analyzer": {},
"char_filter": {},
"filter": {},
"normalizer": {},
"tokenizer": {}
},
"settings": {},
"time_series": {
"": "string"
},
"queries": {
"cache": {
"enabled": true
}
},
"similarity": {},
"mapping": {
"coerce": true,
"total_fields": {
"limit": 42.0,
"ignore_dynamic_beyond_limit": true
},
"depth": {
"limit": 42.0
},
"nested_fields": {
"limit": 42.0
},
"nested_objects": {
"limit": 42.0
},
"field_name_length": {
"limit": 42.0
},
"dimension_fields": {
"limit": 42.0
},
"source": {
"mode": "DISABLED"
},
"ignore_malformed": true
},
"indexing.slowlog": {
"level": "string",
"source": 42.0,
"reformat": true,
"threshold": {
"index": {
"warn": "string",
"info": "string",
"debug": "string",
"trace": "string"
}
}
},
"indexing_pressure": {
"memory": {
"limit": 42.0
}
},
"store": {
"": "fs",
"allow_mmap": true
}
},
"defaults": {
"index": {},
"mode": "string",
"routing_path": "string",
"soft_deletes": {
"enabled": true,
"retention_lease": {
"period": "string"
}
},
"sort": {
"field": "string",
"order": "asc",
"mode": "min",
"missing": "_last"
},
"number_of_shards": 42.0,
"number_of_replicas": 42.0,
"number_of_routing_shards": 42.0,
"check_on_startup": "true",
"codec": "string",
"": "string",
"load_fixed_bitset_filters_eagerly": true,
"hidden": true,
"auto_expand_replicas": "string",
"merge": {
"scheduler": {
"": 42.0
}
},
"search": {
"idle": {
"after": "string"
},
"slowlog": {
"level": "string",
"source": 42.0,
"reformat": true,
"threshold": {
"query": {},
"fetch": {}
}
}
},
"refresh_interval": "string",
"max_result_window": 42.0,
"max_inner_result_window": 42.0,
"max_rescore_window": 42.0,
"max_docvalue_fields_search": 42.0,
"max_script_fields": 42.0,
"max_ngram_diff": 42.0,
"max_shingle_diff": 42.0,
"blocks": {
"": true
},
"max_refresh_listeners": 42.0,
"analyze": {
"": 42.0
},
"highlight": {
"max_analyzed_offset": 42.0
},
"max_terms_count": 42.0,
"max_regex_length": 42.0,
"routing": {
"allocation": {
"enable": "all",
"include": {
"_tier_preference": "string",
"_id": "string"
},
"initial_recovery": {
"_id": "string"
},
"disk": {}
},
"rebalance": {
"enable": "all"
}
},
"gc_deletes": "string",
"default_pipeline": "string",
"final_pipeline": "string",
"lifecycle": {
"name": "string",
"": true,
"origination_date": 42.0,
"parse_origination_date": true,
"step": {
"wait_time_threshold": "string"
},
"rollover_alias": "string",
"prefer_ilm": true
},
"provided_name": "string",
"uuid": "string",
"version": {
"created": "string",
"created_string": "string"
},
"verified_before_close": true,
"format": "string",
"max_slices_per_scroll": 42.0,
"translog": {
"sync_interval": "string",
"durability": "request",
"": 42.0,
"retention": {
"": 42.0,
"age": "string"
}
},
"query_string": {
"": true
},
"priority": 42.0,
"top_metrics_max_size": 42.0,
"analysis": {
"analyzer": {},
"char_filter": {},
"filter": {},
"normalizer": {},
"tokenizer": {}
},
"settings": {},
"time_series": {
"": "string"
},
"queries": {
"cache": {
"enabled": true
}
},
"similarity": {},
"mapping": {
"coerce": true,
"total_fields": {
"limit": 42.0,
"ignore_dynamic_beyond_limit": true
},
"depth": {
"limit": 42.0
},
"nested_fields": {
"limit": 42.0
},
"nested_objects": {
"limit": 42.0
},
"field_name_length": {
"limit": 42.0
},
"dimension_fields": {
"limit": 42.0
},
"source": {
"mode": "DISABLED"
},
"ignore_malformed": true
},
"indexing.slowlog": {
"level": "string",
"source": 42.0,
"reformat": true,
"threshold": {
"index": {
"warn": "string",
"info": "string",
"debug": "string",
"trace": "string"
}
}
},
"indexing_pressure": {
"memory": {
"limit": 42.0
}
},
"store": {
"": "fs",
"allow_mmap": true
}
},
"data_stream": "string",
"lifecycle": {
"data_retention": "string",
"downsampling": {
"rounds": [
{
"after": "string",
"config": {}
}
]
},
"enabled": true
}
}
}
Simulate data ingestion Technical preview
Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.
This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.
The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.
This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.
By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.
Query parameters
-
pipeline string
The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.
Body Required
-
Sample documents to test in the pipeline.
-
A map of component template names to substitute component template definition objects.
-
index_template_subtitutions object
A map of index template names to substitute index template definition objects.
-
mapping_addition object
Additional properties are allowed.
-
pipeline_substitutions object
Pipelines to test. If you don’t specify the
pipeline
request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.
curl \
--request POST http://api.example.com/_ingest/_simulate \
--header "Content-Type: application/json" \
--data '{"docs":[{"_id":123,"_index":"my-index","_source":{"foo":"bar"}},{"_id":456,"_index":"my-index","_source":{"foo":"rab"}}]}'
{
"docs": [
{
"_id": 123,
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": 456,
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
]
}
{
"docs": [
{
"_id": 123,
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": 456,
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
],
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"uppercase": {
"field": "foo"
}
}
]
}
}
}
{
"docs": [
{
"_index": "my-index",
"_id": "123",
"_source": {
"foo": "foo"
}
},
{
"_index": "my-index",
"_id": "456",
"_source": {
"bar": "rab"
}
}
],
"component_template_substitutions": {
"my-mappings_template": {
"template": {
"mappings": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
},
"bar": {
"type": "keyword"
}
}
}
}
}
}
}
{
"docs": [
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
],
"mapping_addition": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
}
}
},
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"set": {
"field": "field3",
"value": "value3"
}
}
]
}
},
"index_template_substitutions": {
"my-index-template": {
"composed_of": [
"component_template_1",
"component_template_2"
],
"index_patterns": [
"my-index-*"
]
}
},
"component_template_substitutions": {
"my-component-template": {
"template": {
"mappings": {
"dynamic": true,
"properties": {
"field3": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"default_pipeline": "my-pipeline"
}
}
}
}
}
}
{
"docs": [
{
"_id": 123,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "bar",
"field1": "value1",
"field2": "value2"
},
"_version": -3,
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"_id": 456,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "rab",
"field1": "value1",
"field2": "value2"
},
"_version": "-3,",
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}
{
"docs": [
{
"_id": 123,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "BAR",
"field2": "value2"
},
"_version": -3,
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"_id": 456,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "RAB",
"field2": "value2"
},
"_version": -3,
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}
{
"docs": [
{
"doc": {
"_id": "123",
"_index": "my-index",
"_version": -3,
"_source": {
"foo": "foo"
},
"executed_pipelines": []
}
},
{
"doc": {
"_id": "456",
"_index": "my-index",
"_version": -3,
"_source": {
"bar": "rab"
},
"executed_pipelines": []
}
}
]
}
Simulate data ingestion Technical preview
Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.
This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.
The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.
This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.
By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.
Path parameters
-
The index to simulate ingesting into. This value can be overridden by specifying an index on each document. If you specify this parameter in the request path, it is used for any documents that do not explicitly specify an index argument.
Query parameters
-
pipeline string
The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.
Body Required
-
Sample documents to test in the pipeline.
-
A map of component template names to substitute component template definition objects.
-
index_template_subtitutions object
A map of index template names to substitute index template definition objects.
-
mapping_addition object
Additional properties are allowed.
-
pipeline_substitutions object
Pipelines to test. If you don’t specify the
pipeline
request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.
curl \
--request GET http://api.example.com/_ingest/{index}/_simulate \
--header "Content-Type: application/json" \
--data '{"docs":[{"_id":123,"_index":"my-index","_source":{"foo":"bar"}},{"_id":456,"_index":"my-index","_source":{"foo":"rab"}}]}'
{
"docs": [
{
"_id": 123,
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": 456,
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
]
}
{
"docs": [
{
"_id": 123,
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": 456,
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
],
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"uppercase": {
"field": "foo"
}
}
]
}
}
}
{
"docs": [
{
"_index": "my-index",
"_id": "123",
"_source": {
"foo": "foo"
}
},
{
"_index": "my-index",
"_id": "456",
"_source": {
"bar": "rab"
}
}
],
"component_template_substitutions": {
"my-mappings_template": {
"template": {
"mappings": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
},
"bar": {
"type": "keyword"
}
}
}
}
}
}
}
{
"docs": [
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "bar"
}
},
{
"_id": "id",
"_index": "my-index",
"_source": {
"foo": "rab"
}
}
],
"mapping_addition": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
}
}
},
"pipeline_substitutions": {
"my-pipeline": {
"processors": [
{
"set": {
"field": "field3",
"value": "value3"
}
}
]
}
},
"index_template_substitutions": {
"my-index-template": {
"composed_of": [
"component_template_1",
"component_template_2"
],
"index_patterns": [
"my-index-*"
]
}
},
"component_template_substitutions": {
"my-component-template": {
"template": {
"mappings": {
"dynamic": true,
"properties": {
"field3": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"default_pipeline": "my-pipeline"
}
}
}
}
}
}
{
"docs": [
{
"_id": 123,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "bar",
"field1": "value1",
"field2": "value2"
},
"_version": -3,
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"_id": 456,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "rab",
"field1": "value1",
"field2": "value2"
},
"_version": "-3,",
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}
{
"docs": [
{
"_id": 123,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "BAR",
"field2": "value2"
},
"_version": -3,
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
},
{
"_id": 456,
"doc": null,
"_index": "my-index",
"_source": {
"foo": "RAB",
"field2": "value2"
},
"_version": -3,
"executed_pipelines": [
"my-pipeline",
"my-final-pipeline"
]
}
]
}
{
"docs": [
{
"doc": {
"_id": "123",
"_index": "my-index",
"_version": -3,
"_source": {
"foo": "foo"
},
"executed_pipelines": []
}
},
{
"doc": {
"_id": "456",
"_index": "my-index",
"_version": -3,
"_source": {
"bar": "rab"
},
"executed_pipelines": []
}
}
]
}
Get machine learning memory usage info Added in 8.2.0
Get information about how machine learning jobs and trained models are using memory, on each node, both within the JVM heap, and natively, outside of the JVM.
Query parameters
-
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET http://api.example.com/_ml/memory/_stats
{
"_nodes": {
"failures": [
{
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
],
"total": 42.0,
"successful": 42.0,
"failed": 42.0
},
"cluster_name": "string",
"nodes": {
"additionalProperty1": {
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"jvm": {
"": 42.0,
"heap_max_in_bytes": 42.0,
"java_inference_in_bytes": 42.0,
"java_inference_max_in_bytes": 42.0
},
"mem": {
"": 42.0,
"adjusted_total_in_bytes": 42.0,
"total_in_bytes": 42.0,
"ml": {
"": 42.0,
"anomaly_detectors_in_bytes": 42.0,
"data_frame_analytics_in_bytes": 42.0,
"max_in_bytes": 42.0,
"native_code_overhead_in_bytes": 42.0,
"native_inference_in_bytes": 42.0
}
},
"name": "string",
"roles": [
"string"
],
"transport_address": "string",
"ephemeral_id": "string"
},
"additionalProperty2": {
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"jvm": {
"": 42.0,
"heap_max_in_bytes": 42.0,
"java_inference_in_bytes": 42.0,
"java_inference_max_in_bytes": 42.0
},
"mem": {
"": 42.0,
"adjusted_total_in_bytes": 42.0,
"total_in_bytes": 42.0,
"ml": {
"": 42.0,
"anomaly_detectors_in_bytes": 42.0,
"data_frame_analytics_in_bytes": 42.0,
"max_in_bytes": 42.0,
"native_code_overhead_in_bytes": 42.0,
"native_inference_in_bytes": 42.0
}
},
"name": "string",
"roles": [
"string"
],
"transport_address": "string",
"ephemeral_id": "string"
}
}
}
Create a datafeed Added in 5.4.0
Datafeeds retrieve data from Elasticsearch for analysis by an anomaly detection job.
You can associate only one datafeed with each anomaly detection job.
The datafeed contains a query that runs at a defined interval (frequency
).
If you are concerned about delayed data, you can add a delay (query_delay') at each interval.
By default, the datafeed uses the following query:
{"match_all": {"boost": 1}}`.
When Elasticsearch security features are enabled, your datafeed remembers which roles the user who created it had
at the time of creation and runs the query using those same roles. If you provide secondary authorization headers,
those credentials are used instead.
You must use Kibana, this API, or the create anomaly detection jobs API to create a datafeed. Do not add a datafeed
directly to the .ml-config
index. Do not give users write
privileges on the .ml-config
index.
Path parameters
-
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Query parameters
-
allow_no_indices boolean
If true, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the
_all
string or when no indices are specified. -
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values.
-
If true, concrete, expanded, or aliased indices are ignored when frozen.
Body Required
-
aggregations object
If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.
-
chunking_config object
Additional properties are allowed.
-
delayed_data_check_config object
Additional properties are allowed.
-
frequency string
A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
indices string | array[string]
-
indices_options object
Additional properties are allowed.
-
job_id string
-
max_empty_searches number
If a real-time datafeed has never seen any data (including during any initial training period), it automatically stops and closes the associated job after this many real-time searches return no documents. In other words, it stops after
frequency
timesmax_empty_searches
of real-time operation. If not set, a datafeed with no end time that sees no data remains started until it is explicitly stopped. By default, it is not set. -
query object
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Additional properties are allowed.
-
query_delay string
A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
runtime_mappings object
-
script_fields object
Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.
-
scroll_size number
The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations. The maximum value is the value of
index.max_result_window
, which is 10,000 by default. -
headers object
curl \
--request PUT http://api.example.com/_ml/datafeeds/{datafeed_id} \
--header "Content-Type: application/json" \
--data '{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0,"headers":{}}'
{
"aggregations": {},
"chunking_config": {
"mode": "auto",
"time_span": "string"
},
"delayed_data_check_config": {
"check_window": "string",
"enabled": true
},
"frequency": "string",
"indices": "string",
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": "string",
"ignore_unavailable": true,
"ignore_throttled": true
},
"job_id": "string",
"max_empty_searches": 42.0,
"query": {},
"query_delay": "string",
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
}
},
"script_fields": {
"additionalProperty1": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
},
"additionalProperty2": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
}
},
"scroll_size": 42.0,
"headers": {}
}
{
"aggregations": {},
"authorization": {
"api_key": {
"id": "string",
"name": "string"
},
"roles": [
"string"
],
"service_account": "string"
},
"chunking_config": {
"mode": "auto",
"time_span": "string"
},
"delayed_data_check_config": {
"check_window": "string",
"enabled": true
},
"datafeed_id": "string",
"frequency": "string",
"indices": [
"string"
],
"job_id": "string",
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": "string",
"ignore_unavailable": true,
"ignore_throttled": true
},
"max_empty_searches": 42.0,
"query": {},
"query_delay": "string",
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
}
},
"script_fields": {
"additionalProperty1": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
},
"additionalProperty2": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
}
},
"scroll_size": 42.0
}
Delete a datafeed Added in 5.4.0
Path parameters
-
A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Query parameters
-
force boolean
Use to forcefully delete a started datafeed; this method is quicker than stopping and deleting the datafeed.
curl \
--request DELETE http://api.example.com/_ml/datafeeds/{datafeed_id}
{
"acknowledged": true
}
Create an anomaly detection job Added in 5.4.0
If you include a datafeed_config
, you must have read index privileges on the source index.
If you include a datafeed_config
but do not provide a query, the datafeed uses {"match_all": {"boost": 1}}
.
Path parameters
-
The identifier for the anomaly detection job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Query parameters
-
allow_no_indices boolean
If
true
, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the_all
string or when no indices are specified. -
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values. Valid values are:
all
: Match any data stream or index, including hidden ones.closed
: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden
: Match hidden data streams and hidden indices. Must be combined withopen
,closed
, or both.none
: Wildcard patterns are not accepted.open
: Match open, non-hidden indices. Also matches any non-hidden data stream.
-
If
true
, concrete, expanded or aliased indices are ignored when frozen.
Body Required
-
allow_lazy_open boolean
Advanced configuration option. Specifies whether this job can open when there is insufficient machine learning node capacity for it to be immediately assigned to a node. By default, if a machine learning node with capacity to run the job cannot immediately be found, the open anomaly detection jobs API returns an error. However, this is also subject to the cluster-wide
xpack.ml.max_lazy_ml_nodes
setting. If this option is set to true, the open anomaly detection jobs API does not return an error and the job waits in the opening state until sufficient machine learning node capacity is available. -
Additional properties are allowed.
-
analysis_limits object
Additional properties are allowed.
-
background_persist_interval string
A duration. Units can be
nanos
,micros
,ms
(milliseconds),s
(seconds),m
(minutes),h
(hours) andd
(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
custom_settings object
Custom metadata about the job
Additional properties are allowed.
-
Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies a period of time (in days) after which only the first snapshot per day is retained. This period is relative to the timestamp of the most recent snapshot for this job. Valid values range from 0 to
model_snapshot_retention_days
. -
Additional properties are allowed.
-
datafeed_config object
Additional properties are allowed.
-
description string
A description of the job.
-
job_id string
-
groups array[string]
A list of job groups. A job can belong to no groups or many.
-
model_plot_config object
Additional properties are allowed.
-
Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job. By default, snapshots ten days older than the newest snapshot are deleted.
-
renormalization_window_days number
Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen. The default value is the longer of 30 days or 100 bucket spans.
-
results_index_name string
-
results_retention_days number
Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained. Annotations generated by the system also count as results for retention purposes; they are deleted after the same number of days as results. Annotations added by users are retained forever.
curl \
--request PUT http://api.example.com/_ml/anomaly_detectors/{job_id} \
--header "Content-Type: application/json" \
--data '{"analysis_config":{"detectors":[{"function":"sum","field_name":"bytes","detector_description":"Sum of bytes"}],"bucket_span":"15m"},"analysis_limits":{"model_memory_limit":"11MB"},"datafeed_config":{"query":{"bool":{"must":[{"match_all":{}}]}},"indices":["kibana_sample_data_logs"],"datafeed_id":"datafeed-test-job1","runtime_mappings":{"hour_of_day":{"type":"long","script":{"source":"emit(doc['timestamp'].value.getHour());"}}}},"data_description":{"time_field":"timestamp","time_format":"epoch_ms"},"model_plot_config":{"enabled":true,"annotations_enabled":true},"results_index_name":"test-job1"}'
{
"analysis_config": {
"detectors": [
{
"function": "sum",
"field_name": "bytes",
"detector_description": "Sum of bytes"
}
],
"bucket_span": "15m"
},
"analysis_limits": {
"model_memory_limit": "11MB"
},
"datafeed_config": {
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
},
"indices": [
"kibana_sample_data_logs"
],
"datafeed_id": "datafeed-test-job1",
"runtime_mappings": {
"hour_of_day": {
"type": "long",
"script": {
"source": "emit(doc['timestamp'].value.getHour());"
}
}
}
},
"data_description": {
"time_field": "timestamp",
"time_format": "epoch_ms"
},
"model_plot_config": {
"enabled": true,
"annotations_enabled": true
},
"results_index_name": "test-job1"
}
{
"job_id": "test-job1",
"job_type": "anomaly_detector",
"create_time": 1656087283340,
"job_version": "8.4.0",
"allow_lazy_open": false,
"analysis_config": {
"detectors": [
{
"function": "sum",
"field_name": "bytes",
"detector_index": 0,
"detector_description": "Sum of bytes"
}
],
"bucket_span": "15m",
"influencers": [],
"model_prune_window": "30d"
},
"analysis_limits": {
"model_memory_limit": "11mb",
"categorization_examples_limit": 4
},
"datafeed_config": {
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
},
"job_id": "test-job1",
"indices": [
"kibana_sample_data_logs"
],
"datafeed_id": "datafeed-test-job1",
"query_delay": "61499ms",
"scroll_size": 1000,
"authorization": {
"roles": [
"superuser"
]
},
"chunking_config": {
"mode": "auto"
},
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": [
"open"
],
"ignore_throttled": true,
"ignore_unavailable": false
},
"runtime_mappings": {
"hour_of_day": {
"type": "long",
"script": {
"source": "emit(doc['timestamp'].value.getHour());"
}
}
},
"delayed_data_check_config": {
"enabled": true
}
},
"data_description": {
"time_field": "timestamp",
"time_format": "epoch_ms"
},
"model_plot_config": {
"enabled": true,
"annotations_enabled": true
},
"results_index_name": "custom-test-job1",
"model_snapshot_retention_days": 10,
"daily_model_snapshot_retention_after_days": 1
}
Preview a datafeed Added in 5.4.0
This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.
Body
-
datafeed_config object
Additional properties are allowed.
-
job_config object
Additional properties are allowed.
curl \
--request POST http://api.example.com/_ml/datafeeds/_preview \
--header "Content-Type: application/json" \
--data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'
{
"datafeed_config": {
"aggregations": {},
"chunking_config": {
"mode": "auto",
"time_span": "string"
},
"datafeed_id": "string",
"delayed_data_check_config": {
"check_window": "string",
"enabled": true
},
"frequency": "string",
"indices": "string",
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": "string",
"ignore_unavailable": true,
"ignore_throttled": true
},
"job_id": "string",
"max_empty_searches": 42.0,
"query": {},
"query_delay": "string",
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
}
},
"script_fields": {
"additionalProperty1": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
},
"additionalProperty2": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
}
},
"scroll_size": 42.0
},
"job_config": {
"allow_lazy_open": true,
"analysis_config": {
"bucket_span": "string",
"": "string",
"categorization_field_name": "string",
"categorization_filters": [
"string"
],
"detectors": [
{
"by_field_name": "string",
"custom_rules": [
{
"actions": [
"skip_result"
],
"conditions": [
{}
],
"scope": {}
}
],
"detector_description": "string",
"detector_index": 42.0,
"exclude_frequent": "all",
"field_name": "string",
"function": "string",
"over_field_name": "string",
"partition_field_name": "string",
"use_null": true
}
],
"influencers": [
"string"
],
"latency": "string",
"model_prune_window": "string",
"multivariate_by_fields": true,
"per_partition_categorization": {
"enabled": true,
"stop_on_warn": true
},
"summary_count_field_name": "string"
},
"analysis_limits": {
"categorization_examples_limit": 42.0,
"": 42.0
},
"background_persist_interval": "string",
"custom_settings": {},
"daily_model_snapshot_retention_after_days": 42.0,
"data_description": {
"format": "string",
"time_field": "string",
"time_format": "string",
"field_delimiter": "string"
},
"datafeed_config": {
"aggregations": {},
"chunking_config": {
"mode": "auto",
"time_span": "string"
},
"datafeed_id": "string",
"delayed_data_check_config": {
"check_window": "string",
"enabled": true
},
"frequency": "string",
"indices": "string",
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": "string",
"ignore_unavailable": true,
"ignore_throttled": true
},
"job_id": "string",
"max_empty_searches": 42.0,
"query": {},
"query_delay": "string",
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
}
},
"script_fields": {
"additionalProperty1": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
},
"additionalProperty2": {
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"ignore_failure": true
}
},
"scroll_size": 42.0
},
"description": "string",
"groups": [
"string"
],
"job_id": "string",
"job_type": "string",
"model_plot_config": {
"annotations_enabled": true,
"enabled": true,
"terms": "string"
},
"model_snapshot_retention_days": 42.0,
"renormalization_window_days": 42.0,
"results_index_name": "string",
"results_retention_days": 42.0
}
}
[
{}
]
Preview features used by data frame analytics Added in 7.13.0
Preview the extracted features used by a data frame analytics config.
curl \
--request GET http://api.example.com/_ml/data_frame/analytics/_preview \
--header "Content-Type: application/json" \
--data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'
{
"config": {
"source": {
"index": "string",
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {
"type": "boolean"
},
"additionalProperty2": {
"type": "boolean"
}
},
"fetch_fields": [
{
"field": "string",
"format": "string"
}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"source": "string",
"id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"": "painless",
"options": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"type": "boolean"
}
},
"_source": {
"includes": [
"string"
],
"excludes": [
"string"
]
},
"query": {}
},
"analysis": {
"": {
"alpha": 42.0,
"dependent_variable": "string",
"downsample_factor": 42.0,
"early_stopping_enabled": true,
"eta": 42.0,
"eta_growth_rate_per_tree": 42.0,
"feature_bag_fraction": 42.0,
"feature_processors": [
{
"frequency_encoding": {},
"multi_encoding": {},
"n_gram_encoding": {},
"one_hot_encoding": {},
"target_mean_encoding": {}
}
],
"gamma": 42.0,
"lambda": 42.0,
"max_optimization_rounds_per_hyperparameter": 42.0,
"max_trees": 42.0,
"num_top_feature_importance_values": 42.0,
"prediction_field_name": "string",
"randomize_seed": 42.0,
"soft_tree_depth_limit": 42.0,
"soft_tree_depth_tolerance": 42.0,
"": "string",
"loss_function": "string",
"loss_function_parameter": 42.0
},
"outlier_detection": {
"compute_feature_influence": true,
"feature_influence_threshold": 42.0,
"method": "string",
"n_neighbors": 42.0,
"outlier_fraction": 42.0,
"standardization_enabled": true
}
},
"model_memory_limit": "string",
"max_num_threads": 42.0,
"analyzed_fields": {
"includes": [
"string"
],
"excludes": [
"string"
]
}
}
}
{
"feature_values": [
{
"additionalProperty1": "string",
"additionalProperty2": "string"
}
]
}
Get rollup job information Deprecated Technical preview
Get the configuration, stats, and status of rollup jobs.
NOTE: This API returns only active (both STARTED
and STOPPED
) jobs.
If a job was created, ran for a while, then was deleted, the API does not return any details about it.
For details about a historical rollup job, the rollup capabilities API may be more useful.
curl \
--request GET http://api.example.com/_rollup/job
{
"jobs": [
{
"config": {
"cron": "string",
"groups": {
"date_histogram": {
"delay": "string",
"field": "string",
"format": "string",
"interval": "string",
"calendar_interval": "string",
"fixed_interval": "string",
"time_zone": "string"
},
"histogram": {
"fields": "string",
"interval": 42.0
},
"terms": {
"fields": "string"
}
},
"id": "string",
"index_pattern": "string",
"metrics": [
{
"field": "string",
"metrics": [
"min"
]
}
],
"page_size": 42.0,
"rollup_index": "string",
"timeout": "string"
},
"stats": {
"documents_processed": 42.0,
"index_failures": 42.0,
"": 42.0,
"index_total": 42.0,
"pages_processed": 42.0,
"rollups_indexed": 42.0,
"search_failures": 42.0,
"search_total": 42.0,
"trigger_count": 42.0,
"processing_total": 42.0
},
"status": {
"current_position": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"job_state": "started",
"upgraded_doc_id": true
}
}
]
}
Get the field capabilities Added in 5.4.0
Get information about the capabilities of fields among multiple indices.
For data streams, the API returns field capabilities among the stream’s backing indices.
It returns runtime fields like any other field.
For example, a runtime field with a type of keyword is returned the same as any other field that belongs to the keyword
family.
Query parameters
-
allow_no_indices boolean
If false, the request returns an error if any wildcard expression, index alias, or
_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*
returns an error if an index starts with foo but no index starts with bar. -
expand_wildcards string | array[string]
The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. -
fields string | array[string]
A comma-separated list of fields to retrieve capabilities for. Wildcard (
*
) expressions are supported. -
include_unmapped boolean
If true, unmapped fields are included in the response.
-
filters string
A comma-separated list of filters to apply to the response.
-
types array[string]
A comma-separated list of field types to include. Any fields that do not match one of these types will be excluded from the results. It defaults to empty, meaning that all field types are returned.
-
include_empty_fields boolean
If false, empty fields are not included in the response.
Body
-
fields string | array[string]
-
index_filter object
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Additional properties are allowed.
-
runtime_mappings object
curl \
--request GET http://api.example.com/_field_caps \
--header "Content-Type: application/json" \
--data '"{\n \"index_filter\": {\n \"range\": {\n \"@timestamp\": {\n \"gte\": \"2018\"\n }\n }\n }\n}"'
{
"index_filter": {
"range": {
"@timestamp": {
"gte": "2018"
}
}
}
}
{
"indices": [ "index1", "index2", "index3", "index4", "index5" ],
"fields": {
"rating": {
"long": {
"metadata_field": false,
"searchable": true,
"aggregatable": false,
"indices": [ "index1", "index2" ],
"non_aggregatable_indices": [ "index1" ]
},
"keyword": {
"metadata_field": false,
"searchable": false,
"aggregatable": true,
"indices": [ "index3", "index4" ],
"non_searchable_indices": [ "index4" ]
}
},
"title": {
"text": {
"metadata_field": false,
"searchable": true,
"aggregatable": false
}
}
}
}
{
"indices": [ "index1", "index2", "index3", "index4", "index5" ],
"fields": {
"rating": {
"long": {
"metadata_field": false,
"searchable": true,
"aggregatable": false,
"indices": [ "index1", "index2" ],
"non_aggregatable_indices": [ "index1" ]
},
"keyword": {
"metadata_field": false,
"searchable": false,
"aggregatable": true,
"indices": [ "index3", "index4" ],
"non_searchable_indices": [ "index4" ]
}
},
"title": {
"text": {
"metadata_field": false,
"searchable": true,
"aggregatable": false
}
}
}
}
Run a knn search Deprecated Technical preview
NOTE: The kNN search API has been replaced by the knn
option in the search API.
Perform a k-nearest neighbor (kNN) search on a dense_vector field and return the matching documents. Given a query vector, the API finds the k closest vectors and returns those documents as search hits.
Elasticsearch uses the HNSW algorithm to support efficient kNN search. Like most kNN algorithms, HNSW is an approximate method that sacrifices result accuracy for improved search speed. This means the results returned are not always the true k closest neighbors.
The kNN search API supports restricting the search using a filter. The search will return the top k documents that also match the filter query.
A kNN search response has the exact same structure as a search API response. However, certain sections have a meaning specific to kNN search:
- The document
_score
is determined by the similarity between the query and document vector. - The
hits.total
object contains the total number of nearest neighbor candidates considered, which isnum_candidates * num_shards
. Thehits.total.relation
will always beeq
, indicating an exact value.
Path parameters
-
A comma-separated list of index names to search; use
_all
or to perform the operation on all indices.
Query parameters
-
routing string
A comma-separated list of specific routing values.
Body
_source boolean | object
Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.
-
docvalue_fields array[object]
The request returns doc values for field names matching these patterns in the
hits.fields
property of the response. It accepts wildcard (*
) patterns. -
stored_fields string | array[string]
-
fields string | array[string]
filter object | array[object]
A query to filter the documents that can match. The kNN search will return the top
k
documents that also match this filter. The value can be a single query or a list of queries. Iffilter
isn't provided, all documents are allowed to match.One of: An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Additional properties are allowed.
-
Additional properties are allowed.
curl \
--request POST http://api.example.com/{index}/_knn_search \
--header "Content-Type: application/json" \
--data '{"":true,"docvalue_fields":[{"field":"string","format":"string","include_unmapped":true}],"stored_fields":"string","fields":"string","filter":{},"knn":{"field":"string","query_vector":[42.0],"k":42.0,"num_candidates":42.0}}'
{
"": true,
"docvalue_fields": [
{
"field": "string",
"format": "string",
"include_unmapped": true
}
],
"stored_fields": "string",
"fields": "string",
"filter": {},
"knn": {
"field": "string",
"query_vector": [
42.0
],
"k": 42.0,
"num_candidates": 42.0
}
}
{
"took": 42.0,
"timed_out": true,
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
},
"hits": {
"total": {
"relation": "eq",
"value": 42.0
},
"hits": [
{
"_index": "string",
"_id": "string",
"_score": 42.0,
"_explanation": {
"description": "string",
"details": [
{}
],
"value": 42.0
},
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"highlight": {
"additionalProperty1": [
"string"
],
"additionalProperty2": [
"string"
]
},
"inner_hits": {
"additionalProperty1": {
"hits": {}
},
"additionalProperty2": {
"hits": {}
}
},
"matched_queries": [
"string"
],
"_nested": {
"field": "string",
"offset": 42.0,
"_nested": {}
},
"_ignored": [
"string"
],
"ignored_field_values": {
"additionalProperty1": [
{}
],
"additionalProperty2": [
{}
]
},
"_shard": "string",
"_node": "string",
"_routing": "string",
"_source": {},
"_rank": 42.0,
"_seq_no": 42.0,
"_primary_term": 42.0,
"_version": 42.0,
"sort": [
42.0
]
}
],
"max_score": 42.0
},
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"max_score": 42.0
}
Evaluate ranked search results Added in 6.2.0
Evaluate the quality of ranked search results over a set of typical search queries.
Query parameters
-
allow_no_indices boolean
If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*
returns an error if an index starts withfoo
but no index starts withbar
. -
expand_wildcards string | array[string]
Whether to expand wildcard expression to concrete indices that are open, closed or both.
-
search_type string
Search operation type
curl \
--request POST http://api.example.com/_rank_eval \
--header "Content-Type: application/json" \
--data '{"requests":[{"id":"string","request":{"query":{},"size":42.0},"ratings":[{"_id":"string","_index":"string","rating":42.0}],"template_id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}}}],"metric":{"":{"k":42.0,"maximum_relevance":42.0}}}'
{
"requests": [
{
"id": "string",
"request": {
"query": {},
"size": 42.0
},
"ratings": [
{
"_id": "string",
"_index": "string",
"rating": 42.0
}
],
"template_id": "string",
"params": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
],
"metric": {
"": {
"k": 42.0,
"maximum_relevance": 42.0
}
}
}
{
"metric_score": 42.0,
"details": {
"additionalProperty1": {
"metric_score": 42.0,
"unrated_docs": [
{
"_id": "string",
"_index": "string"
}
],
"hits": [
{
"hit": {
"_id": "string",
"_index": "string",
"_score": 42.0
},
"rating": 42.0
}
],
"metric_details": {
"additionalProperty1": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"additionalProperty2": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
},
"additionalProperty2": {
"metric_score": 42.0,
"unrated_docs": [
{
"_id": "string",
"_index": "string"
}
],
"hits": [
{
"hit": {
"_id": "string",
"_index": "string",
"_score": 42.0
},
"rating": 42.0
}
],
"metric_details": {
"additionalProperty1": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"additionalProperty2": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
}
},
"failures": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
Create a service account token
Create a service accounts token for access without requiring basic authentication.
NOTE: Service account tokens never expire. You must actively delete them if they are no longer needed.
Path parameters
-
The name of the namespace, which is a top-level grouping of service accounts.
-
The name of the service.
-
The name for the service account token. If omitted, a random name will be generated.
Token names must be at least one and no more than 256 characters. They can contain alphanumeric characters (a-z, A-Z, 0-9), dashes (
-
), and underscores (_
), but cannot begin with an underscore.NOTE: Token names must be unique in the context of the associated service account. They must also be globally unique with their fully qualified names, which are comprised of the service account principal and token name, such as
<namespace>/<service>/<token-name>
.
Query parameters
-
refresh string
If
true
then refresh the affected shards to make this operation visible to search, ifwait_for
(the default) then wait for a refresh to make this operation visible to search, iffalse
then do nothing with refreshes.Values are
true
,false
, orwait_for
.
curl \
--request POST http://api.example.com/_security/service/{namespace}/{service}/credential/token/{name}
{
"created": true,
"token": {
"name": "string",
"value": "string"
}
}
Check user profile privileges Added in 8.3.0
Determine whether the users associated with the specified user profile IDs have all the requested privileges.
NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.
Body Required
-
A list of profile IDs. The privileges are checked for associated users of the profiles.
-
Additional properties are allowed.
curl \
--request POST http://api.example.com/_security/profile/_has_privileges \
--header "Content-Type: application/json" \
--data '{"uids":["string"],"privileges":{"application":[{"application":"string","privileges":["string"],"resources":["string"]}],"cluster":["string"],"index":[{"names":"string","privileges":["string"],"allow_restricted_indices":true}]}}'
{
"uids": [
"string"
],
"privileges": {
"application": [
{
"application": "string",
"privileges": [
"string"
],
"resources": [
"string"
]
}
],
"cluster": [
"string"
],
"index": [
{
"names": "string",
"privileges": [
"string"
],
"allow_restricted_indices": true
}
]
}
}
{
"has_privilege_uids": [
"string"
],
"errors": {
"count": 42.0,
"details": {
"*": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
}
}
}
Logout of SAML completely Added in 7.14.0
Verifies the logout response sent from the SAML IdP.
NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.
The SAML IdP may send a logout response back to the SP after handling the SP-initiated SAML Single Logout. This API verifies the response by ensuring the content is relevant and validating its signature. An empty response is returned if the verification process is successful. The response can be sent by the IdP with either the HTTP-Redirect or the HTTP-Post binding. The caller of this API must prepare the request accordingly so that this API can handle either of them.
Body Required
-
The name of the SAML realm in Elasticsearch for which the configuration is used to verify the logout response.
ids string | array[string] Required
-
query_string string
If the SAML IdP sends the logout response with the HTTP-Redirect binding, this field must be set to the query string of the redirect URI.
-
content string
If the SAML IdP sends the logout response with the HTTP-Post binding, this field must be set to the value of the SAMLResponse form parameter from the logout response.
curl \
--request POST http://api.example.com/_security/saml/complete_logout \
--header "Content-Type: application/json" \
--data '{"realm":"string","":"string","query_string":"string","content":"string"}'
{
"realm": "string",
"": "string",
"query_string": "string",
"content": "string"
}
Run a policy Added in 7.4.0
Immediately create a snapshot according to the snapshot lifecycle policy without waiting for the scheduled time. The snapshot policy is normally applied according to its schedule, but you might want to manually run a policy before performing an upgrade or other maintenance.
Path parameters
-
The id of the snapshot lifecycle policy to be executed
Query parameters
-
master_timeout string
The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout string
The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request PUT http://api.example.com/_slm/policy/{policy_id}/_execute
{
"snapshot_name": "string"
}
Stop snapshot lifecycle management Added in 7.6.0
Stop all snapshot lifecycle management (SLM) operations and the SLM plugin. This API is useful when you are performing maintenance on a cluster and need to prevent SLM from performing any actions on your data streams or indices. Stopping SLM does not stop any snapshots that are in progress. You can manually trigger snapshots with the run snapshot lifecycle policy API even if SLM is stopped.
The API returns a response as soon as the request is acknowledged, but the plugin might continue to run until in-progress operations complete and it can be safely stopped. Use the get snapshot lifecycle management status API to see if SLM is running.
Query parameters
-
master_timeout string
The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to
-1
. -
timeout string
The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to
-1
.
curl \
--request POST http://api.example.com/_slm/stop
{
"acknowledged": true
}
Query parameters
-
allow_no_match boolean
Specifies what to do when the request:
- Contains wildcard expressions and there are no transforms that match.
- Contains the _all string or no identifiers and there are no matches.
- Contains wildcard expressions and there are only partial matches.
If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.
-
from number
Skips the specified number of transforms.
-
size number
Specifies the maximum number of transforms to obtain.
-
exclude_generated boolean
Excludes fields that were automatically added when creating the transform. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET http://api.example.com/_transform
{
"count": 42.0,
"transforms": [
{
"authorization": {
"api_key": {
"id": "string",
"name": "string"
},
"roles": [
"string"
],
"service_account": "string"
},
"": 42.0,
"description": "string",
"dest": {
"index": "string",
"op_type": "index",
"pipeline": "string",
"routing": "string",
"version_type": "internal"
},
"frequency": "string",
"id": "string",
"latest": {
"sort": "string",
"unique_key": [
"string"
]
},
"pivot": {
"aggregations": {},
"group_by": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"retention_policy": {
"time": {
"field": "string",
"max_age": "string"
}
},
"settings": {
"align_checkpoints": true,
"dates_as_epoch_millis": true,
"deduce_mappings": true,
"docs_per_second": 42.0,
"max_page_search_size": 42.0,
"unattended": true
},
"source": {
"index": "string",
"runtime_mappings": {
"additionalProperty1": {
"fields": {},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {},
"type": "boolean"
},
"additionalProperty2": {
"fields": {},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {},
"type": "boolean"
}
},
"query": {}
},
"sync": {
"time": {
"delay": "string",
"field": "string"
}
},
"version": "string",
"_meta": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
]
}