- Elasticsearch Guide: other versions:
- Elasticsearch introduction
- Getting started with Elasticsearch
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- All permission check
- Starting Elasticsearch
- Stopping Elasticsearch
- Adding nodes to your cluster
- Installing X-Pack
- Set up X-Pack
- Configuring X-Pack Java Clients
- X-Pack Settings
- Bootstrap Checks for X-Pack
- Upgrade Elasticsearch
- API Conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Weighted Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top Hits Aggregation
- Value Count Aggregation
- Median Absolute Deviation Aggregation
- Bucket Aggregations
- Adjacency Matrix Aggregation
- Auto-interval Date Histogram Aggregation
- Children Aggregation
- Composite Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Parent Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Moving Function Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Bucket Sort Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Split Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Scripting
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Normalizers
- Tokenizers
- Standard Tokenizer
- Letter Tokenizer
- Lowercase Tokenizer
- Whitespace Tokenizer
- UAX URL Email Tokenizer
- Classic Tokenizer
- Thai Tokenizer
- NGram Tokenizer
- Edge NGram Tokenizer
- Keyword Tokenizer
- Pattern Tokenizer
- Char Group Tokenizer
- Simple Pattern Tokenizer
- Simple Pattern Split Tokenizer
- Path Hierarchy Tokenizer
- Path Hierarchy Tokenizer Examples
- Token Filters
- Standard Token Filter
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Multiplexer Token Filter
- Conditional Token Filter
- Predicate Token Filter Script
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Parsing synonym files
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Exclude mode settings example
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- MinHash Token Filter
- Remove Duplicates Token Filter
- Character Filters
- Modules
- Index Modules
- Ingest Node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Conditional Execution in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Bytes Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Dissect Processor
- Dot Expander Processor
- Drop Processor
- Fail Processor
- Foreach Processor
- GeoIP Processor
- Grok Processor
- Gsub Processor
- Join Processor
- JSON Processor
- KV Processor
- Lowercase Processor
- Pipeline Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Set Security User Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- URL Decode Processor
- User Agent processor
- Managing the index lifecycle
- SQL Access
- Monitor a cluster
- Rolling up historical data
- Frozen indices
- Set up a cluster for high availability
- Secure a cluster
- Overview
- Configuring security
- Encrypting communications in Elasticsearch
- Encrypting communications in an Elasticsearch Docker Container
- Enabling cipher suites for stronger encryption
- Separating node-to-node and client traffic
- Configuring an Active Directory realm
- Configuring a file realm
- Configuring an LDAP realm
- Configuring a native realm
- Configuring a PKI realm
- Configuring a SAML realm
- Configuring a Kerberos realm
- FIPS 140-2
- Security settings
- Security files
- Auditing Settings
- How security works
- User authentication
- Built-in users
- Internal users
- Token-based authentication services
- Realms
- Realm chains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- User authorization
- Auditing security events
- Encrypting communications
- Restricting connections with IP filtering
- Cross cluster search, tribe, clients, and integrations
- Tutorial: Getting started with security
- Tutorial: Encrypting communications
- Troubleshooting
- Can’t log in after upgrading to 6.8.23
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Alerting on Cluster and Index Events
- Command line tools
- How To
- Glossary of terms
- X-Pack APIs
- Info API
- Cross-cluster replication APIs
- Explore API
- Freeze index
- Index lifecycle management API
- Licensing APIs
- Migration APIs
- Machine learning APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create calendar
- Create datafeeds
- Create filter
- Create jobs
- Delete calendar
- Delete datafeeds
- Delete events from calendar
- Delete filter
- Delete forecast
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Delete expired data
- Find file structure
- Flush jobs
- Forecast jobs
- Get calendars
- Get buckets
- Get overall buckets
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get machine learning info
- Get model snapshots
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Revert model snapshots
- Set upgrade mode
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filter
- Update jobs
- Update model snapshots
- Rollup APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Create API keys
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Create or update users
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete users
- Disable users
- Enable users
- Get API key information
- Get application privileges
- Get role mappings
- Get roles
- Get token
- Get users
- Has privileges
- Invalidate API key
- Invalidate token
- SSL certificate
- Unfreeze index
- Watcher APIs
- Definitions
- Release Highlights
- Breaking changes
- Release Notes
- Elasticsearch version 6.8.23
- Elasticsearch version 6.8.22
- Elasticsearch version 6.8.21
- Elasticsearch version 6.8.20
- Elasticsearch version 6.8.19
- Elasticsearch version 6.8.18
- Elasticsearch version 6.8.17
- Elasticsearch version 6.8.16
- Elasticsearch version 6.8.15
- Elasticsearch version 6.8.14
- Elasticsearch version 6.8.13
- Elasticsearch version 6.8.12
- Elasticsearch version 6.8.11
- Elasticsearch version 6.8.10
- Elasticsearch version 6.8.9
- Elasticsearch version 6.8.8
- Elasticsearch version 6.8.7
- Elasticsearch version 6.8.6
- Elasticsearch version 6.8.5
- Elasticsearch version 6.8.4
- Elasticsearch version 6.8.3
- Elasticsearch version 6.8.2
- Elasticsearch version 6.8.1
- Elasticsearch version 6.8.0
- Elasticsearch version 6.7.2
- Elasticsearch version 6.7.1
- Elasticsearch version 6.7.0
- Elasticsearch version 6.6.2
- Elasticsearch version 6.6.1
- Elasticsearch version 6.6.0
- Elasticsearch version 6.5.4
- Elasticsearch version 6.5.3
- Elasticsearch version 6.5.2
- Elasticsearch version 6.5.1
- Elasticsearch version 6.5.0
- Elasticsearch version 6.4.3
- Elasticsearch version 6.4.2
- Elasticsearch version 6.4.1
- Elasticsearch version 6.4.0
- Elasticsearch version 6.3.2
- Elasticsearch version 6.3.1
- Elasticsearch version 6.3.0
- Elasticsearch version 6.2.4
- Elasticsearch version 6.2.3
- Elasticsearch version 6.2.2
- Elasticsearch version 6.2.1
- Elasticsearch version 6.2.0
- Elasticsearch version 6.1.4
- Elasticsearch version 6.1.3
- Elasticsearch version 6.1.2
- Elasticsearch version 6.1.1
- Elasticsearch version 6.1.0
- Elasticsearch version 6.0.1
- Elasticsearch version 6.0.0
- Elasticsearch version 6.0.0-rc2
- Elasticsearch version 6.0.0-rc1
- Elasticsearch version 6.0.0-beta2
- Elasticsearch version 6.0.0-beta1
- Elasticsearch version 6.0.0-alpha2
- Elasticsearch version 6.0.0-alpha1
- Elasticsearch version 6.0.0-alpha1 (Changes previously released in 5.x)
NOTE: You are looking at documentation for an older release. For the latest information, see the current release documentation.
Sort
editSort
editAllows you to add one or more sorts on specific fields. Each sort can be
reversed as well. The sort is defined on a per field level, with special
field name for _score
to sort by score, and _doc
to sort by index order.
Assuming the following index mapping:
PUT /my_index { "mappings": { "_doc": { "properties": { "post_date": { "type": "date" }, "user": { "type": "keyword" }, "name": { "type": "keyword" }, "age": { "type": "integer" } } } } }
GET /my_index/_search { "sort" : [ { "post_date" : {"order" : "asc"}}, "user", { "name" : "desc" }, { "age" : "desc" }, "_score" ], "query" : { "term" : { "user" : "kimchy" } } }
_doc
has no real use-case besides being the most efficient sort order.
So if you don’t care about the order in which documents are returned, then you
should sort by _doc
. This especially helps when scrolling.
Sort Values
editThe sort values for each document returned are also returned as part of the response.
Sort Order
editThe order
option can have the following values:
|
Sort in ascending order |
|
Sort in descending order |
The order defaults to desc
when sorting on the _score
, and defaults
to asc
when sorting on anything else.
Sort mode option
editElasticsearch supports sorting by array or multi-valued fields. The mode
option
controls what array value is picked for sorting the document it belongs
to. The mode
option can have the following values:
|
Pick the lowest value. |
|
Pick the highest value. |
|
Use the sum of all values as sort value. Only applicable for number based array fields. |
|
Use the average of all values as sort value. Only applicable for number based array fields. |
|
Use the median of all values as sort value. Only applicable for number based array fields. |
The default sort mode in the ascending sort order is min
— the lowest value
is picked. The default sort mode in the descending order is max
— the highest value is picked.
Sort mode example usage
editIn the example below the field price has multiple prices per document. In this case the result hits will be sorted by price ascending based on the average price per document.
PUT /my_index/_doc/1?refresh { "product": "chocolate", "price": [20, 4] } POST /_search { "query" : { "term" : { "product" : "chocolate" } }, "sort" : [ {"price" : {"order" : "asc", "mode" : "avg"}} ] }
Sorting within nested objects.
editElasticsearch also supports sorting by
fields that are inside one or more nested objects. The sorting by nested
field support has a nested
sort option with the following properties:
-
path
- Defines on which nested object to sort. The actual sort field must be a direct field inside this nested object. When sorting by nested field, this field is mandatory.
-
filter
-
A filter that the inner objects inside the nested path
should match with in order for its field values to be taken into account
by sorting. Common case is to repeat the query / filter inside the
nested filter or query. By default no
nested_filter
is active. -
max_children
- The maximum number of children to consider per root document when picking the sort value. Defaults to unlimited.
-
nested
-
Same as top-level
nested
but applies to another nested path within the current nested object.
Nested sort options before Elasticseach 6.1
The nested_path
and nested_filter
options have been deprecated in
favor of the options documented above.
Nested sorting examples
editIn the below example offer
is a field of type nested
.
The nested path
needs to be specified; otherwise, Elasticsearch doesn’t know on what nested level sort values need to be captured.
POST /_search { "query" : { "term" : { "product" : "chocolate" } }, "sort" : [ { "offer.price" : { "mode" : "avg", "order" : "asc", "nested": { "path": "offer", "filter": { "term" : { "offer.color" : "blue" } } } } } ] }
In the below example parent
and child
fields are of type nested
.
The nested_path
needs to be specified at each level; otherwise, Elasticsearch doesn’t know on what nested level sort values need to be captured.
POST /_search { "query": { "nested": { "path": "parent", "query": { "bool": { "must": {"range": {"parent.age": {"gte": 21}}}, "filter": { "nested": { "path": "parent.child", "query": {"match": {"parent.child.name": "matt"}} } } } } } }, "sort" : [ { "parent.child.age" : { "mode" : "min", "order" : "asc", "nested": { "path": "parent", "filter": { "range": {"parent.age": {"gte": 21}} }, "nested": { "path": "parent.child", "filter": { "match": {"parent.child.name": "matt"} } } } } } ] }
Nested sorting is also supported when sorting by scripts and sorting by geo distance.
Missing Values
editThe missing
parameter specifies how docs which are missing
the sort field should be treated: The missing
value can be
set to _last
, _first
, or a custom value (that
will be used for missing docs as the sort value).
The default is _last
.
For example:
GET /_search { "sort" : [ { "price" : {"missing" : "_last"} } ], "query" : { "term" : { "product" : "chocolate" } } }
If a nested inner object doesn’t match with
the nested_filter
then a missing value is used.
Ignoring Unmapped Fields
editBy default, the search request will fail if there is no mapping
associated with a field. The unmapped_type
option allows you to ignore
fields that have no mapping and not sort by them. The value of this
parameter is used to determine what sort values to emit. Here is an
example of how it can be used:
GET /_search { "sort" : [ { "price" : {"unmapped_type" : "long"} } ], "query" : { "term" : { "product" : "chocolate" } } }
If any of the indices that are queried doesn’t have a mapping for price
then Elasticsearch will handle it as if there was a mapping of type
long
, with all documents in this index having no value for this field.
Geo Distance Sorting
editAllow to sort by _geo_distance
. Here is an example, assuming pin.location
is a field of type geo_point
:
GET /_search { "sort" : [ { "_geo_distance" : { "pin.location" : [-70, 40], "order" : "asc", "unit" : "km", "mode" : "min", "distance_type" : "arc", "ignore_unmapped": true } } ], "query" : { "term" : { "user" : "kimchy" } } }
-
distance_type
-
How to compute the distance. Can either be
arc
(default), orplane
(faster, but inaccurate on long distances and close to the poles). -
mode
-
What to do in case a field has several geo points. By default, the shortest
distance is taken into account when sorting in ascending order and the
longest distance when sorting in descending order. Supported values are
min
,max
,median
andavg
. -
unit
-
The unit to use when computing sort values. The default is
m
(meters). -
ignore_unmapped
-
Indicates if the unmapped field should be treated as a missing value. Setting it to
true
is equivalent to specifying anunmapped_type
in the field sort. The default isfalse
(unmapped field cause the search to fail).
geo distance sorting does not support configurable missing values: the
distance will always be considered equal to Infinity
when a document does not
have values for the field that is used for distance computation.
The following formats are supported in providing the coordinates:
Lat Lon as Properties
editGET /_search { "sort" : [ { "_geo_distance" : { "pin.location" : { "lat" : 40, "lon" : -70 }, "order" : "asc", "unit" : "km" } } ], "query" : { "term" : { "user" : "kimchy" } } }
Lat Lon as String
editFormat in lat,lon
.
GET /_search { "sort" : [ { "_geo_distance" : { "pin.location" : "40,-70", "order" : "asc", "unit" : "km" } } ], "query" : { "term" : { "user" : "kimchy" } } }
Geohash
editGET /_search { "sort" : [ { "_geo_distance" : { "pin.location" : "drm3btev3e86", "order" : "asc", "unit" : "km" } } ], "query" : { "term" : { "user" : "kimchy" } } }
Multiple reference points
editMultiple geo points can be passed as an array containing any geo_point
format, for example
GET /_search { "sort" : [ { "_geo_distance" : { "pin.location" : [[-70, 40], [-71, 42]], "order" : "asc", "unit" : "km" } } ], "query" : { "term" : { "user" : "kimchy" } } }
and so forth.
The final distance for a document will then be min
/max
/avg
(defined via mode
) distance of all points contained in the document to all points given in the sort request.
Script Based Sorting
editAllow to sort based on custom scripts, here is an example:
GET /_search { "query" : { "term" : { "user" : "kimchy" } }, "sort" : { "_script" : { "type" : "number", "script" : { "lang": "painless", "source": "doc['field_name'].value * params.factor", "params" : { "factor" : 1.1 } }, "order" : "asc" } } }
Track Scores
editWhen sorting on a field, scores are not computed. By setting
track_scores
to true, scores will still be computed and tracked.
GET /_search { "track_scores": true, "sort" : [ { "post_date" : {"order" : "desc"} }, { "name" : "desc" }, { "age" : "desc" } ], "query" : { "term" : { "user" : "kimchy" } } }
Memory Considerations
editWhen sorting, the relevant sorted field values are loaded into memory.
This means that per shard, there should be enough memory to contain
them. For string based types, the field sorted on should not be analyzed
/ tokenized. For numeric types, if possible, it is recommended to
explicitly set the type to narrower types (like short
, integer
and
float
).
On this page
- Sort Values
- Sort Order
- Sort mode option
- Sort mode example usage
- Sorting within nested objects.
- Nested sorting examples
- Missing Values
- Ignoring Unmapped Fields
- Geo Distance Sorting
- Lat Lon as Properties
- Lat Lon as String
- Geohash
- Lat Lon as Array
- Multiple reference points
- Script Based Sorting
- Track Scores
- Memory Considerations