Get snapshot information Added in 2.1.0
Get information about the snapshots stored in one or more repositories. A snapshot is a backup of an index or running Elasticsearch cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get snapshot API.
Path parameters
-
A comma-separated list of snapshot repositories used to limit the request. Accepts wildcard expressions.
_all
returns all repositories. If any repository fails during the request, Elasticsearch returns an error.
Query parameters
-
master_timeout string
Period to wait for a connection to the master node.
-
time string
Unit used to display time values.
Values are
nanos
,micros
,ms
,s
,m
,h
, ord
.
curl \
--request GET http://api.example.com/_cat/snapshots/{repository}
[
{
"id": "string",
"repository": "string",
"status": "string",
"": 42.0,
"end_time": "string",
"duration": "string",
"indices": "string",
"successful_shards": "string",
"failed_shards": "string",
"total_shards": "string",
"reason": "string"
}
]
id repository status start_epoch start_time end_epoch end_time duration indices successful_shards failed_shards total_shards
snap1 repo1 FAILED 1445616705 18:11:45 1445616978 18:16:18 4.6m 1 4 1 5
snap2 repo1 SUCCESS 1445634298 23:04:58 1445634672 23:11:12 6.2m 2 10 0 10
Get the cluster state Added in 1.3.0
Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true
query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
Path parameters
-
Limit the information returned to the specified metrics
Query parameters
-
allow_no_indices boolean
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes
_all
string or when no indices have been specified) -
expand_wildcards string | array[string]
Whether to expand wildcard expression to concrete indices that are open, closed or both.
-
flat_settings boolean
Return settings in flat format (default: false)
-
local boolean
Return local information, do not retrieve the state from master node (default: false)
-
master_timeout string
Specify timeout for connection to master
-
wait_for_metadata_version number
Wait for the metadata version to be equal or greater than the specified metadata version
-
wait_for_timeout string
The maximum time to wait for wait_for_metadata_version before timing out
curl \
--request GET http://api.example.com/_cluster/state/{metric}
{}
Get cluster statistics Added in 1.3.0
Get basic index metrics (shard numbers, store size, memory usage) and information about the current nodes that form the cluster (number, roles, os, jvm versions, memory usage, cpu and installed plugins).
Path parameters
-
Comma-separated list of node filters used to limit returned information. Defaults to all nodes in the cluster.
Query parameters
-
include_remotes boolean
Include remote cluster data into the response
-
timeout string
Period to wait for each node to respond. If a node does not respond before its timeout expires, the response does not include its stats. However, timed out nodes are included in the response’s
_nodes.failed
property. Defaults to no timeout.
curl \
--request GET http://api.example.com/_cluster/stats/nodes/{node_id}
{
"_nodes": {
"failures": [
{
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
],
"total": 42.0,
"successful": 42.0,
"failed": 42.0
},
"cluster_name": "string",
"cluster_uuid": "string",
"indices": {
"analysis": {
"analyzer_types": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"built_in_analyzers": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"built_in_char_filters": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"built_in_filters": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"built_in_tokenizers": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"char_filter_types": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"filter_types": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"tokenizer_types": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
]
},
"completion": {
"size_in_bytes": 42.0,
"": 42.0,
"fields": {
"additionalProperty1": {
"size_in_bytes": 42.0
},
"additionalProperty2": {
"size_in_bytes": 42.0
}
}
},
"count": 42.0,
"docs": {
"count": 42.0,
"deleted": 42.0
},
"fielddata": {
"evictions": 42.0,
"": 42.0,
"memory_size_in_bytes": 42.0,
"fields": {
"additionalProperty1": {
"memory_size_in_bytes": 42.0
},
"additionalProperty2": {
"memory_size_in_bytes": 42.0
}
}
},
"query_cache": {
"cache_count": 42.0,
"cache_size": 42.0,
"evictions": 42.0,
"hit_count": 42.0,
"": 42.0,
"memory_size_in_bytes": 42.0,
"miss_count": 42.0,
"total_count": 42.0
},
"segments": {
"count": 42.0,
"": 42.0,
"doc_values_memory_in_bytes": 42.0,
"file_sizes": {
"additionalProperty1": {
"description": "string",
"size_in_bytes": 42.0,
"min_size_in_bytes": 42.0,
"max_size_in_bytes": 42.0,
"average_size_in_bytes": 42.0,
"count": 42.0
},
"additionalProperty2": {
"description": "string",
"size_in_bytes": 42.0,
"min_size_in_bytes": 42.0,
"max_size_in_bytes": 42.0,
"average_size_in_bytes": 42.0,
"count": 42.0
}
},
"fixed_bit_set_memory_in_bytes": 42.0,
"index_writer_max_memory_in_bytes": 42.0,
"index_writer_memory_in_bytes": 42.0,
"max_unsafe_auto_id_timestamp": 42.0,
"memory_in_bytes": 42.0,
"norms_memory_in_bytes": 42.0,
"points_memory_in_bytes": 42.0,
"stored_fields_memory_in_bytes": 42.0,
"terms_memory_in_bytes": 42.0,
"term_vectors_memory_in_bytes": 42.0,
"version_map_memory_in_bytes": 42.0
},
"shards": {
"index": {
"primaries": {
"avg": 42.0,
"max": 42.0,
"min": 42.0
},
"replication": {
"avg": 42.0,
"max": 42.0,
"min": 42.0
},
"shards": {
"avg": 42.0,
"max": 42.0,
"min": 42.0
}
},
"primaries": 42.0,
"replication": 42.0,
"total": 42.0
},
"store": {
"": 42.0,
"size_in_bytes": 42.0,
"reserved_in_bytes": 42.0,
"total_data_set_size_in_bytes": 42.0
},
"mappings": {
"field_types": [
{
"name": "string",
"count": 42.0,
"index_count": 42.0,
"indexed_vector_count": 42.0,
"indexed_vector_dim_max": 42.0,
"indexed_vector_dim_min": 42.0,
"script_count": 42.0
}
],
"runtime_field_types": [
{
"chars_max": 42.0,
"chars_total": 42.0,
"count": 42.0,
"doc_max": 42.0,
"doc_total": 42.0,
"index_count": 42.0,
"lang": [
"string"
],
"lines_max": 42.0,
"lines_total": 42.0,
"name": "string",
"scriptless_count": 42.0,
"shadowed_count": 42.0,
"source_max": 42.0,
"source_total": 42.0
}
],
"total_field_count": 42.0,
"total_deduplicated_field_count": 42.0,
"": 42.0,
"total_deduplicated_mapping_size_in_bytes": 42.0
},
"versions": [
{
"index_count": 42.0,
"primary_shard_count": 42.0,
"total_primary_bytes": 42.0,
"version": "string"
}
]
},
"nodes": {
"count": {
"coordinating_only": 42.0,
"data": 42.0,
"data_cold": 42.0,
"data_content": 42.0,
"data_frozen": 42.0,
"data_hot": 42.0,
"data_warm": 42.0,
"ingest": 42.0,
"master": 42.0,
"ml": 42.0,
"remote_cluster_client": 42.0,
"total": 42.0,
"transform": 42.0,
"voting_only": 42.0
},
"discovery_types": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"fs": {
"available_in_bytes": 42.0,
"free_in_bytes": 42.0,
"total_in_bytes": 42.0
},
"indexing_pressure": {
"memory": {
"current": {
"all_in_bytes": 42.0,
"combined_coordinating_and_primary_in_bytes": 42.0,
"coordinating_in_bytes": 42.0,
"coordinating_rejections": 42.0,
"primary_in_bytes": 42.0,
"primary_rejections": 42.0,
"replica_in_bytes": 42.0,
"replica_rejections": 42.0
},
"limit_in_bytes": 42.0,
"total": {
"all_in_bytes": 42.0,
"combined_coordinating_and_primary_in_bytes": 42.0,
"coordinating_in_bytes": 42.0,
"coordinating_rejections": 42.0,
"primary_in_bytes": 42.0,
"primary_rejections": 42.0,
"replica_in_bytes": 42.0,
"replica_rejections": 42.0
}
}
},
"ingest": {
"number_of_pipelines": 42.0,
"processor_stats": {
"additionalProperty1": {
"count": 42.0,
"current": 42.0,
"failed": 42.0,
"time": "string"
},
"additionalProperty2": {
"count": 42.0,
"current": 42.0,
"failed": 42.0,
"time": "string"
}
}
},
"jvm": {
"": 42.0,
"mem": {
"heap_max_in_bytes": 42.0,
"heap_used_in_bytes": 42.0
},
"threads": 42.0,
"versions": [
{
"bundled_jdk": true,
"count": 42.0,
"using_bundled_jdk": true,
"version": "string",
"vm_name": "string",
"vm_vendor": "string",
"vm_version": "string"
}
]
},
"network_types": {
"http_types": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"transport_types": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
}
},
"os": {
"allocated_processors": 42.0,
"architectures": [
{
"arch": "string",
"count": 42.0
}
],
"available_processors": 42.0,
"mem": {
"adjusted_total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"free_percent": 42.0,
"total_in_bytes": 42.0,
"used_in_bytes": 42.0,
"used_percent": 42.0
},
"names": [
{
"count": 42.0,
"name": "string"
}
],
"pretty_names": [
{
"count": 42.0,
"pretty_name": "string"
}
]
},
"packaging_types": [
{
"count": 42.0,
"flavor": "string",
"type": "string"
}
],
"plugins": [
{
"classname": "string",
"description": "string",
"elasticsearch_version": "string",
"extended_plugins": [
"string"
],
"has_native_controller": true,
"java_version": "string",
"name": "string",
"version": "string",
"licensed": true
}
],
"process": {
"cpu": {
"percent": 42.0
},
"open_file_descriptors": {
"avg": 42.0,
"max": 42.0,
"min": 42.0
}
},
"versions": [
"string"
]
},
"status": "green",
"timestamp": 42.0
}
Get feature usage information Added in 6.0.0
Path parameters
-
Limits the information returned to the specific metrics. A comma-separated list of the following options:
_all
,rest_actions
.
Query parameters
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET http://api.example.com/_nodes/usage/{metric}
{
"_nodes": {
"failures": [
{
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
],
"total": 42.0,
"successful": 42.0,
"failed": 42.0
},
"cluster_name": "string",
"nodes": {
"additionalProperty1": {
"rest_actions": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"": 42.0,
"aggregations": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"additionalProperty2": {
"rest_actions": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"": 42.0,
"aggregations": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
}
}
Pause an auto-follow pattern Added in 7.5.0
Pause a cross-cluster replication auto-follow pattern. When the API returns, the auto-follow pattern is inactive. New indices that are created on the remote cluster and match the auto-follow patterns are ignored.
You can resume auto-following with the resume auto-follow pattern API. When it resumes, the auto-follow pattern is active again and automatically configures follower indices for newly created indices on the remote cluster that match its patterns. Remote indices that were created while the pattern was paused will also be followed, unless they have been deleted or closed in the interim.
Path parameters
-
The name of the auto-follow pattern to pause.
Query parameters
-
master_timeout string
The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to
-1
to indicate that the request should never timeout.
curl \
--request POST http://api.example.com/_ccr/auto_follow/{name}/pause
{
"acknowledged" : true
}
Resume an auto-follow pattern Added in 7.5.0
Resume a cross-cluster replication auto-follow pattern that was paused. The auto-follow pattern will resume configuring following indices for newly created indices that match its patterns on the remote cluster. Remote indices created while the pattern was paused will also be followed unless they have been deleted or closed in the interim.
Path parameters
-
The name of the auto-follow pattern to resume.
Query parameters
-
master_timeout string
The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to
-1
to indicate that the request should never timeout.
curl \
--request POST http://api.example.com/_ccr/auto_follow/{name}/resume
{
"acknowledged" : true
}
Create or update a document in an index
Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.
NOTE: You cannot use this API to send update requests for existing documents in a data stream.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
- To add or overwrite a document using the
PUT /<target>/_doc/<_id>
request format, you must have thecreate
,index
, orwrite
index privilege. - To add a document using the
POST /<target>/_doc/
request format, you must have thecreate_doc
,create
,index
, orwrite
index privilege. - To automatically create a data stream or index with this API request, you must have the
auto_configure
,create_index
, ormanage
index privilege.
Automatic data stream creation requires a matching index template with data stream enabled.
NOTE: Replica shards might not all be started when an indexing operation returns successfully.
By default, only the primary is required. Set wait_for_active_shards
to change this default behavior.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream
definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index
setting.
If it is true
, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false
to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with +
or -
to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index
setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Optimistic concurrency control
Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no
and if_primary_term
parameters.
If a mismatch is detected, the operation will result in a VersionConflictException
and a status code of 409
.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing
parameter.
When setting up explicit mapping, you can also use the _routing
field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing
mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing
setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards
is 1
).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards
.
To alter this behavior per operation, use the wait_for_active_shards request
parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas
+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards
is set on the request to 3
(and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards
to all
(or to 4
, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards
section of the API response reveals the number of shard copies on which replication succeeded and failed.
No operation (noop) updates
When updating a document by using this API, a new version of the document is always created even if the document hasn't changed.
If this isn't acceptable use the _update
API with detect_noop
set to true
.
The detect_noop
option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.
There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.
Versioning
Each indexed document is given a version number.
By default, internal versioning is used that starts at 1 and increments with each update, deletes included.
Optionally, the version number can be set to an external value (for example, if maintained in a database).
To enable this functionality, version_type
should be set to external
.
The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18
.
NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.
When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:
PUT my-index-000001/_doc/1?version=2&version_type=external
{
"user": {
"id": "elkbee"
}
}
In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).
A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
Path parameters
-
The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (
*
) pattern of an index template with adata_stream
definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API. -
A unique identifier for the document. To automatically generate a document ID, use the
POST /<target>/_doc/
request format and omit this parameter.
Query parameters
-
if_primary_term number
Only perform the operation if the document has this primary term.
-
if_seq_no number
Only perform the operation if the document has this sequence number.
-
include_source_on_error boolean
True or false if to include the document source in the error message in case of parsing errors.
-
op_type string
Set to
create
to only index the document if it does not already exist (put if absent). If a document with the specified_id
already exists, the indexing operation will fail. The behavior is the same as using the<index>/_create
endpoint. If a document ID is specified, this paramater defaults toindex
. Otherwise, it defaults tocreate
. If the request targets a data stream, anop_type
ofcreate
is required.Values are
index
orcreate
. -
pipeline string
The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to
_none
disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter. -
refresh string
If
true
, Elasticsearch refreshes the affected shards to make this operation visible to search. Ifwait_for
, it waits for a refresh to make this operation visible to search. Iffalse
, it does nothing with refreshes.Values are
true
,false
, orwait_for
. -
routing string
A custom value that is used to route operations to a specific shard.
-
timeout string
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
-
version number
An explicit version number for concurrency control. It must be a non-negative long number.
-
version_type string
The version type.
Values are
internal
,external
,external_gte
, orforce
. -
wait_for_active_shards number | string
The number of shard copies that must be active before proceeding with the operation. You can set it to
all
or any positive integer up to the total number of shards in the index (number_of_replicas+1
). The default value of1
means it waits for each primary shard to be active. -
require_alias boolean
If
true
, the destination must be an index alias.
curl \
--request POST http://api.example.com/{index}/_doc/{id} \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "W0tpsmIBdwcYyG50zbta",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}
{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}
Create or update a document in an index
Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.
NOTE: You cannot use this API to send update requests for existing documents in a data stream.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
- To add or overwrite a document using the
PUT /<target>/_doc/<_id>
request format, you must have thecreate
,index
, orwrite
index privilege. - To add a document using the
POST /<target>/_doc/
request format, you must have thecreate_doc
,create
,index
, orwrite
index privilege. - To automatically create a data stream or index with this API request, you must have the
auto_configure
,create_index
, ormanage
index privilege.
Automatic data stream creation requires a matching index template with data stream enabled.
NOTE: Replica shards might not all be started when an indexing operation returns successfully.
By default, only the primary is required. Set wait_for_active_shards
to change this default behavior.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream
definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index
setting.
If it is true
, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false
to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with +
or -
to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index
setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Optimistic concurrency control
Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no
and if_primary_term
parameters.
If a mismatch is detected, the operation will result in a VersionConflictException
and a status code of 409
.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing
parameter.
When setting up explicit mapping, you can also use the _routing
field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing
mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing
setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards
is 1
).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards
.
To alter this behavior per operation, use the wait_for_active_shards request
parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas
+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards
is set on the request to 3
(and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards
to all
(or to 4
, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards
section of the API response reveals the number of shard copies on which replication succeeded and failed.
No operation (noop) updates
When updating a document by using this API, a new version of the document is always created even if the document hasn't changed.
If this isn't acceptable use the _update
API with detect_noop
set to true
.
The detect_noop
option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.
There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.
Versioning
Each indexed document is given a version number.
By default, internal versioning is used that starts at 1 and increments with each update, deletes included.
Optionally, the version number can be set to an external value (for example, if maintained in a database).
To enable this functionality, version_type
should be set to external
.
The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18
.
NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.
When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:
PUT my-index-000001/_doc/1?version=2&version_type=external
{
"user": {
"id": "elkbee"
}
}
In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).
A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
Path parameters
-
The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (
*
) pattern of an index template with adata_stream
definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.
Query parameters
-
if_primary_term number
Only perform the operation if the document has this primary term.
-
if_seq_no number
Only perform the operation if the document has this sequence number.
-
include_source_on_error boolean
True or false if to include the document source in the error message in case of parsing errors.
-
op_type string
Set to
create
to only index the document if it does not already exist (put if absent). If a document with the specified_id
already exists, the indexing operation will fail. The behavior is the same as using the<index>/_create
endpoint. If a document ID is specified, this paramater defaults toindex
. Otherwise, it defaults tocreate
. If the request targets a data stream, anop_type
ofcreate
is required.Values are
index
orcreate
. -
pipeline string
The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to
_none
disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter. -
refresh string
If
true
, Elasticsearch refreshes the affected shards to make this operation visible to search. Ifwait_for
, it waits for a refresh to make this operation visible to search. Iffalse
, it does nothing with refreshes.Values are
true
,false
, orwait_for
. -
routing string
A custom value that is used to route operations to a specific shard.
-
timeout string
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
-
version number
An explicit version number for concurrency control. It must be a non-negative long number.
-
version_type string
The version type.
Values are
internal
,external
,external_gte
, orforce
. -
wait_for_active_shards number | string
The number of shard copies that must be active before proceeding with the operation. You can set it to
all
or any positive integer up to the total number of shards in the index (number_of_replicas+1
). The default value of1
means it waits for each primary shard to be active. -
require_alias boolean
If
true
, the destination must be an index alias.
curl \
--request POST http://api.example.com/{index}/_doc \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "W0tpsmIBdwcYyG50zbta",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}
{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}
Get multiple documents Added in 1.3.0
Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
Filter source fields
By default, the _source
field is returned for every document (if stored).
Use the _source
and _source_include
or source_exclude
attributes to filter what fields are returned for a particular document.
You can include the _source
, _source_includes
, and _source_excludes
query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
Use the stored_fields
attribute to specify the set of stored fields you want to retrieve.
Any requested fields that are not stored are ignored.
You can include the stored_fields
query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Query parameters
-
force_synthetic_source boolean
Should this request force synthetic _source? Use this to test if the mapping supports synthetic _source and to get a sense of the worst case performance. Fetches with this enabled will be slower the enabling synthetic source natively in the index.
-
preference string
Specifies the node or shard the operation should be performed on. Random by default.
-
realtime boolean
If
true
, the request is real-time as opposed to near-real-time. -
refresh boolean
If
true
, the request refreshes relevant shards before retrieving documents. -
routing string
Custom value used to route operations to a specific shard.
-
_source boolean | string | array[string]
True or false to return the
_source
field or not, or a list of fields to return. -
_source_excludes string | array[string]
A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in
_source_includes
query parameter. -
_source_includes string | array[string]
A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the
_source_excludes
query parameter. If the_source
parameter isfalse
, this parameter is ignored. -
stored_fields string | array[string]
If
true
, retrieves the document fields stored in the index rather than the document_source
.
Body Required
curl \
--request POST http://api.example.com/_mget \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"1\"\n },\n {\n \"_id\": \"2\"\n }\n ]\n}"'
{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}
{
"docs": [
{
"_index": "test",
"_id": "1",
"_source": false
},
{
"_index": "test",
"_id": "2",
"_source": [ "field3", "field4" ]
},
{
"_index": "test",
"_id": "3",
"_source": {
"include": [ "user" ],
"exclude": [ "user.location" ]
}
}
]
}
{
"docs": [
{
"_index": "test",
"_id": "1",
"stored_fields": [ "field1", "field2" ]
},
{
"_index": "test",
"_id": "2",
"stored_fields": [ "field3", "field4" ]
}
]
}
{
"docs": [
{
"_index": "test",
"_id": "1",
"routing": "key2"
},
{
"_index": "test",
"_id": "2"
}
]
}
{
"docs": [
{
"_index": "string",
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"_ignored": [
"string"
],
"found": true,
"_id": "string",
"_primary_term": 42.0,
"_routing": "string",
"_seq_no": 42.0,
"_source": {},
"_version": 42.0
}
]
}
Get multiple documents Added in 1.3.0
Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.
Filter source fields
By default, the _source
field is returned for every document (if stored).
Use the _source
and _source_include
or source_exclude
attributes to filter what fields are returned for a particular document.
You can include the _source
, _source_includes
, and _source_excludes
query parameters in the request URI to specify the defaults to use when there are no per-document instructions.
Get stored fields
Use the stored_fields
attribute to specify the set of stored fields you want to retrieve.
Any requested fields that are not stored are ignored.
You can include the stored_fields
query parameter in the request URI to specify the defaults to use when there are no per-document instructions.
Path parameters
-
Name of the index to retrieve documents from when
ids
are specified, or when a document in thedocs
array does not specify an index.
Query parameters
-
force_synthetic_source boolean
Should this request force synthetic _source? Use this to test if the mapping supports synthetic _source and to get a sense of the worst case performance. Fetches with this enabled will be slower the enabling synthetic source natively in the index.
-
preference string
Specifies the node or shard the operation should be performed on. Random by default.
-
realtime boolean
If
true
, the request is real-time as opposed to near-real-time. -
refresh boolean
If
true
, the request refreshes relevant shards before retrieving documents. -
routing string
Custom value used to route operations to a specific shard.
-
_source boolean | string | array[string]
True or false to return the
_source
field or not, or a list of fields to return. -
_source_excludes string | array[string]
A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in
_source_includes
query parameter. -
_source_includes string | array[string]
A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the
_source_excludes
query parameter. If the_source
parameter isfalse
, this parameter is ignored. -
stored_fields string | array[string]
If
true
, retrieves the document fields stored in the index rather than the document_source
.
Body Required
curl \
--request GET http://api.example.com/{index}/_mget \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"1\"\n },\n {\n \"_id\": \"2\"\n }\n ]\n}"'
{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}
{
"docs": [
{
"_index": "test",
"_id": "1",
"_source": false
},
{
"_index": "test",
"_id": "2",
"_source": [ "field3", "field4" ]
},
{
"_index": "test",
"_id": "3",
"_source": {
"include": [ "user" ],
"exclude": [ "user.location" ]
}
}
]
}
{
"docs": [
{
"_index": "test",
"_id": "1",
"stored_fields": [ "field1", "field2" ]
},
{
"_index": "test",
"_id": "2",
"stored_fields": [ "field3", "field4" ]
}
]
}
{
"docs": [
{
"_index": "test",
"_id": "1",
"routing": "key2"
},
{
"_index": "test",
"_id": "2"
}
]
}
{
"docs": [
{
"_index": "string",
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"_ignored": [
"string"
],
"found": true,
"_id": "string",
"_primary_term": 42.0,
"_routing": "string",
"_seq_no": 42.0,
"_source": {},
"_version": 42.0
}
]
}
Get multiple term vectors
Get multiple term vectors with a single request.
You can specify existing documents by index and ID or provide artificial documents in the body of the request.
You can specify the index in the request body or request URI.
The response contains a docs
array with all the fetched termvectors.
Each element has the structure provided by the termvectors API.
Artificial documents
You can also use mtermvectors
to generate term vectors for artificial documents provided in the body of the request.
The mapping used is determined by the specified _index
.
Query parameters
-
ids array[string]
A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body
-
fields string | array[string]
A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the
completion_fields
orfielddata_fields
parameters. -
field_statistics boolean
If
true
, the response includes the document count, sum of document frequencies, and sum of total term frequencies. -
offsets boolean
If
true
, the response includes term offsets. -
payloads boolean
If
true
, the response includes term payloads. -
positions boolean
If
true
, the response includes term positions. -
preference string
The node or shard the operation should be performed on. It is random by default.
-
realtime boolean
If true, the request is real-time as opposed to near-real-time.
-
routing string
A custom value used to route operations to a specific shard.
-
term_statistics boolean
If true, the response includes term frequency and document frequency.
-
version number
If
true
, returns the document version as part of a hit. -
version_type string
The version type.
Values are
internal
,external
,external_gte
, orforce
.
curl \
--request GET http://api.example.com/_mtermvectors \
--header "Content-Type: application/json" \
--data '"{\n \"docs\": [\n {\n \"_id\": \"2\",\n \"fields\": [\n \"message\"\n ],\n \"term_statistics\": true\n },\n {\n \"_id\": \"1\"\n }\n ]\n}"'
{
"docs": [
{
"_id": "2",
"fields": [
"message"
],
"term_statistics": true
},
{
"_id": "1"
}
]
}
{
"ids": [ "1", "2" ],
"parameters": {
"fields": [
"message"
],
"term_statistics": true
}
}
{
"docs": [
{
"_index": "my-index-000001",
"doc" : {
"message" : "test test test"
}
},
{
"_index": "my-index-000001",
"doc" : {
"message" : "Another test ..."
}
}
]
}
{
"docs": [
{
"_id": "string",
"_index": "string",
"_version": 42.0,
"took": 42.0,
"found": true,
"term_vectors": {
"additionalProperty1": {
"field_statistics": {
"doc_count": 42.0,
"sum_doc_freq": 42.0,
"sum_ttf": 42.0
},
"terms": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"additionalProperty2": {
"field_statistics": {
"doc_count": 42.0,
"sum_doc_freq": 42.0,
"sum_ttf": 42.0
},
"terms": {
"additionalProperty1": {},
"additionalProperty2": {}
}
}
},
"error": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
}
]
}
Throttle a reindex operation Added in 2.4.0
Change the number of requests per second for a particular reindex operation. For example:
POST _reindex/r1A2WoRbTwKZ516z6NEs5A:36619/_rethrottle?requests_per_second=-1
Rethrottling that speeds up the query takes effect immediately. Rethrottling that slows down the query will take effect after completing the current batch. This behavior prevents scroll timeouts.
Path parameters
-
The task identifier, which can be found by using the tasks API.
Query parameters
-
requests_per_second number
The throttle for this request in sub-requests per second. It can be either
-1
to turn off throttling or any decimal number like1.7
or12
to throttle to that level.
curl \
--request POST http://api.example.com/_reindex/{task_id}/_rethrottle
{
"nodes": {}
}
Get term vector information
Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields
parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime
parameter to false
.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
- term frequency in the field (always returned)
- term positions (
positions: true
) - start and end offsets (
offsets: true
) - term payloads (
payloads: true
), as base64 encoded bytes
If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing
only to hit a particular shard.
Path parameters
-
The name of the index that contains the document.
Query parameters
-
fields string | array[string]
A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the
completion_fields
orfielddata_fields
parameters. -
field_statistics boolean
If
true
, the response includes:- The document count (how many documents contain this field).
- The sum of document frequencies (the sum of document frequencies for all terms in this field).
- The sum of total term frequencies (the sum of total term frequencies of each term in this field).
-
offsets boolean
If
true
, the response includes term offsets. -
payloads boolean
If
true
, the response includes term payloads. -
positions boolean
If
true
, the response includes term positions. -
preference string
The node or shard the operation should be performed on. It is random by default.
-
realtime boolean
If true, the request is real-time as opposed to near-real-time.
-
routing string
A custom value that is used to route operations to a specific shard.
-
term_statistics boolean
If
true
, the response includes:- The total term frequency (how often a term occurs in all documents).
- The document frequency (the number of documents containing the current term).
By default these values are not returned since term statistics can have a serious performance impact.
-
version number
If
true
, returns the document version as part of a hit. -
version_type string
The version type.
Values are
internal
,external
,external_gte
, orforce
.
Body
-
doc object
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Additional properties are allowed.
-
filter object
Additional properties are allowed.
-
per_field_analyzer object
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
curl \
--request GET http://api.example.com/{index}/_termvectors \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'
{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}
{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}
{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}
{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}
{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}
{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}
{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}
{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}
Delete indices
Deleting an index deletes its documents, shards, and metadata. It does not delete related Kibana components, such as data views, visualizations, or dashboards.
You cannot delete the current write index of a data stream. To delete the index, you must roll over the data stream so a new write index is created. You can then use the delete index API to delete the previous write index.
Path parameters
-
Comma-separated list of indices to delete. You cannot specify index aliases. By default, this parameter does not support wildcards (
*
) or_all
. To use wildcards or_all
, set theaction.destructive_requires_name
cluster setting tofalse
.
Query parameters
-
allow_no_indices boolean
If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
. -
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE http://api.example.com/{index}
{
"acknowledged": true,
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
}
}
Path parameters
-
Comma-separated list of data streams or indices to add. Supports wildcards (
*
). Wildcard patterns that match both data streams and indices return an error. -
Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.
Query parameters
-
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Body
-
filter object
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Additional properties are allowed.
-
index_routing string
-
is_write_index boolean
If
true
, sets the write index or data stream for the alias. If an alias points to multiple indices or data streams andis_write_index
isn’t set, the alias rejects write requests. If an index alias points to one index andis_write_index
isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream. -
routing string
-
search_routing string
curl \
--request POST http://api.example.com/{index}/_alias/{name} \
--header "Content-Type: application/json" \
--data '{"filter":{},"index_routing":"string","is_write_index":true,"routing":"string","search_routing":"string"}'
{
"filter": {},
"index_routing": "string",
"is_write_index": true,
"routing": "string",
"search_routing": "string"
}
{
"acknowledged": true
}
Create or update an index template Added in 7.9.0
Index templates define settings, mappings, and aliases that can be applied automatically to new indices.
Elasticsearch applies templates to new indices based on an wildcard pattern that matches the index name. Index templates are applied during data stream or index creation. For data streams, these settings and mappings are applied when the stream's backing indices are created. Settings and mappings specified in a create index API request override any settings or mappings specified in an index template. Changes to index templates do not affect existing indices, including the existing backing indices of a data stream.
You can use C-style /* *\/
block comments in index templates.
You can include comments anywhere in the request body, except before the opening curly bracket.
Multiple matching templates
If multiple index templates match the name of a new index or data stream, the template with the highest priority is used.
Multiple templates with overlapping index patterns at the same priority are not allowed and an error will be thrown when attempting to create a template matching an existing index template at identical priorities.
Composing aliases, mappings, and settings
When multiple component templates are specified in the composed_of
field for an index template, they are merged in the order specified, meaning that later component templates override earlier component templates.
Any mappings, settings, or aliases from the parent index template are merged in next.
Finally, any configuration on the index request itself is merged.
Mapping definitions are merged recursively, which means that later mapping components can introduce new field mappings and update the mapping configuration.
If a field mapping is already contained in an earlier component, its definition will be completely overwritten by the later one.
This recursive merging strategy applies not only to field mappings, but also root options like dynamic_templates
and meta
.
If an earlier component contains a dynamic_templates
block, then by default new dynamic_templates
entries are appended onto the end.
If an entry already exists with the same key, then it is overwritten by the new definition.
Path parameters
-
Index or template name
Query parameters
-
create boolean
If
true
, this request cannot replace or update existing index templates. -
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
cause string
User defined reason for creating/updating the index template
Body Required
-
index_patterns string | array[string]
-
composed_of array[string]
An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
-
template object
Additional properties are allowed.
-
data_stream object
Additional properties are allowed.
-
priority number
Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
-
version number
-
_meta object
-
allow_auto_create boolean
This setting overrides the value of the
action.auto_create_index
cluster setting. If set totrue
in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled viaactions.auto_create_index
. If set tofalse
, then indices or data streams matching the template must always be explicitly created, and may never be automatically created. -
ignore_missing_component_templates array[string]
The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
-
deprecated boolean
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request POST http://api.example.com/_index_template/{name} \
--header "Content-Type: application/json" \
--data '"{\n \"index_patterns\" : [\"template*\"],\n \"priority\" : 1,\n \"template\": {\n \"settings\" : {\n \"number_of_shards\" : 2\n }\n }\n}"'
{
"index_patterns" : ["template*"],
"priority" : 1,
"template": {
"settings" : {
"number_of_shards" : 2
}
}
}
{
"acknowledged": true
}
Delete a legacy index template
Path parameters
-
The name of the legacy index template to delete. Wildcard (
*
) expressions are supported.
Query parameters
-
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request DELETE http://api.example.com/_template/{name}
{
"acknowledged": true
}
Check existence of index templates
Get information about whether index templates exist. Index templates define settings, mappings, and aliases that can be applied automatically to new indices.
IMPORTANT: This documentation is about legacy index templates, which are deprecated and will be replaced by the composable templates introduced in Elasticsearch 7.8.
Path parameters
-
A comma-separated list of index template names used to limit the request. Wildcard (
*
) expressions are supported.
Query parameters
-
flat_settings boolean
Indicates whether to use a flat format for the response.
-
local boolean
Indicates whether to get information from the local node only.
-
master_timeout string
The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicate that the request should never timeout, set it to
-1
.
curl \
--request HEAD http://api.example.com/_template/{name}
Query parameters
-
allow_no_indices boolean
If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
expand_wildcards string | array[string]
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
. -
master_timeout string
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET http://api.example.com/_alias
{
"additionalProperty1": {
"aliases": {
"additionalProperty1": {
"filter": {},
"index_routing": "string",
"is_write_index": true,
"routing": "string",
"search_routing": "string",
"is_hidden": true
},
"additionalProperty2": {
"filter": {},
"index_routing": "string",
"is_write_index": true,
"routing": "string",
"search_routing": "string",
"is_hidden": true
}
}
},
"additionalProperty2": {
"aliases": {
"additionalProperty1": {
"filter": {},
"index_routing": "string",
"is_write_index": true,
"routing": "string",
"search_routing": "string",
"is_hidden": true
},
"additionalProperty2": {
"filter": {},
"index_routing": "string",
"is_write_index": true,
"routing": "string",
"search_routing": "string",
"is_hidden": true
}
}
}
}
Move to a lifecycle step Added in 6.6.0
Manually move an index into a specific step in the lifecycle policy and run that step.
WARNING: This operation can result in the loss of data. Manually moving an index into a specific step runs that step even if it has already been performed. This is a potentially destructive action and this should be considered an expert level API.
You must specify both the current step and the step to be executed in the body of the request. The request will fail if the current step does not match the step currently running for the index This is to prevent the index from being moved from an unexpected step into the next step.
When specifying the target (next_step
) to which the index will be moved, either the name or both the action and name fields are optional.
If only the phase is specified, the index will move to the first step of the first action in the target phase.
If the phase and action are specified, the index will move to the first step of the specified action in the specified phase.
Only actions specified in the ILM policy are considered valid.
An index cannot move to a step that is not part of its policy.
Path parameters
-
The name of the index whose lifecycle step is to change
Body
-
Additional properties are allowed.
-
Additional properties are allowed.
curl \
--request POST http://api.example.com/_ilm/move/{index} \
--header "Content-Type: application/json" \
--data '"{\n \"current_step\": {\n \"phase\": \"new\",\n \"action\": \"complete\",\n \"name\": \"complete\"\n },\n \"next_step\": {\n \"phase\": \"warm\",\n \"action\": \"forcemerge\",\n \"name\": \"forcemerge\"\n }\n}"'
{
"current_step": {
"phase": "new",
"action": "complete",
"name": "complete"
},
"next_step": {
"phase": "warm",
"action": "forcemerge",
"name": "forcemerge"
}
}
{
"current_step": {
"phase": "hot",
"action": "complete",
"name": "complete"
},
"next_step": {
"phase": "warm"
}
}
{
"acknowledged": true
}
Inference
Inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.