Elasticsearch version 6.2.0

edit

Breaking Changes

edit
Aggregations
  • Add a new cluster setting to limit the total number of buckets returned by a request #27581 (issues: #26012, #27452)
Core
  • Forbid granting the all permission in production #27548
Highlighting
  • Limit the analyzed text for highlighting #27934 (issue: #27517)
Rollover
  • Fail rollover if duplicated alias found in templates #28110 (issue: #26976)
Search
  • Introduce limit to the number of terms in Terms Query #27968 (issue: #18829)

Breaking Java Changes

edit
Java API
  • Remove operationThreaded from Java API #27836
Java High Level REST Client
  • REST high-level client: remove index suffix from indices client method names #28263

Deprecations

edit
Analysis
  • Backport delimited payload filter renaming #27535 (issue: #26625)
Suggesters
  • deprecating jarowinkler in favor of jaro_winkler #27526
  • Deprecating levenstein in favor of levensHtein #27409 (issue: #27325)

New Features

edit
Plugin Ingest GeoIp
  • Enable ASN support for Ingest GeoIP plugin. #27958 (issue: #27849)
Plugin Lang Painless
  • Painless: Add spi jar that will be published for extending whitelists #28302
  • Painless: Add a simple cache for whitelist methods and fields. #28142
Plugins
  • Add the ability to bundle multiple plugins into a meta plugin #28022 (issue: #27316)
Rank Evaluation
  • Backport of ranking evaluation API (#27478) #27844 (issue: #27478)
Recovery
  • Backport for using lastSyncedGlobalCheckpoint in deletion policy #27866 (issue: #27826)
Reindex API
  • Add scroll parameter to _reindex API #28041 (issue: #27555)

Enhancements

edit
Allocation
  • Fix cluster.routing.allocation.enable and cluster.routing.rebalance.enable case #28037 (issue: #28007)
  • Add node id to shard failure message #28024 (issue: #28018)
Analysis
  • Limit the analyzed text for highlighting (#27934) #28176 (issue: #27517)
  • Allow TrimFilter to be used in custom normalizers #27758 (issue: #27310)
Circuit Breakers
  • Add accounting circuit breaker and track segment memory usage #27116 (issue: #27044)
Cluster
  • Adds wait_for_no_initializing_shards to cluster health API #27489 (issue: #25623)
Core
Discovery
  • Add information when master node left to DiscoveryNodes' shortSummary() #28197 (issue: #28169)
Engine
  • Move uid lock into LiveVersionMap #27905
  • Optimize version map for append-only indexing #27752
Geo
  • [GEO] Add WKT Support to GeoBoundingBoxQueryBuilder #27692 (issues: #27690, #9120)
  • [Geo] Add Well Known Text (WKT) Parsing Support to ShapeBuilders #27417 (issue: #9120)
Highlighting
  • Include all sentences smaller than fragment_size in the unified highlighter #28132 (issue: #28089)
Ingest
Internal
  • Make KeyedLock reentrant #27920
  • Make AbstractQueryBuilder.declareStandardFields to be protected (#27865) #27894 (issue: #27865)
  • Tighten the CountedBitSet class #27632
  • Avoid doing redundant work when checking for self references. #26927 (issue: #26907)
Java API
  • Add missing delegate methods to NodeIndicesStats #28092
  • Java api clean-up : consistency for shards_acknowledged getters #27819 (issue: #27784)
Java High Level REST Client
  • add toString implementation for UpdateRequest. #27997 (issue: #27986)
  • Add Close Index API to the high level REST client #27734 (issue: #27205)
  • Add Open Index API to the high level REST client #27574 (issue: #27205)
  • Added Create Index support to high-level REST client #27351 (issue: #27205)
  • Add multi get api to the high level rest client #27337 (issue: #27205)
  • Add msearch api to high level client #27274
Mapping
Network
  • Add NioGroup for use in different transports #27737 (issue: #27260)
  • Add read timeouts to http module #27713
  • Implement byte array reusage in NioTransport #27696 (issue: #27563)
  • Introduce resizable inbound byte buffer #27551 (issue: #27563)
  • Decouple nio constructs from the tcp transport #27484 (issue: #27260)
Packaging
Percolator
  • also extract match_all queries when indexing percolator queries #27585
Plugin Lang Painless
  • Painless: Add whitelist extensions #28161
  • Painless: Modify Loader to Load Classes Directly from Definition #28088
  • Clean Up Painless Cast Object #27794
  • Painless: Only allow Painless type names to be the same as the equivalent Java class. #27264
Plugins
  • Add client actions to action plugin #28280 (issue: #27759)
  • Plugins: Add validation to plugin descriptor parsing #27951
  • Plugins: Add plugin extension capabilities #27881
  • Add support for filtering mappings fields #27603
Rank Evaluation
  • Simplify RankEvalResponse output #28266
Recovery
  • Truncate tlog cli should assign global checkpoint #28192 (issue: #28181)
  • Replica starts peer recovery with safe commit #28181 (issue: #10708)
  • Primary send safe commit in file-based recovery #28038 (issue: #10708)
  • Fail resync-failed shards in subsequent writes #28005
  • Introduce promoting index shard state #28004 (issue: #24841)
  • Non-peer recovery should set the global checkpoint #27965
  • Persist global checkpoint when finalizing a peer recovery #27947 (issue: #27861)
  • Rollback a primary before recovering from translog #27804 (issue: #10708)
Search
  • Use typeName() to check field type in GeoShapeQueryBuilder #27730
  • Optimize search_after when sorting in index sort order #26401
Sequence IDs
  • Do not keep 5.x commits when having 6.x commits #28188 (issues: #27606, #28038)
  • Use lastSyncedGlobalCheckpoint in deletion policy #27826 (issue: #27606)
  • Use CountedBitSet in LocalCheckpointTracker #27793
  • Only fsync global checkpoint if needed #27652
  • Keep commits and translog up to the global checkpoint #27606
  • Adjust CombinedDeletionPolicy for multiple commits #27456 (issues: #10708, #27367)
  • Keeps index commits up to the current global checkpoint #27367 (issue: #10708)
  • Dedup translog operations by reading in reverse #27268 (issue: #10708)
Settings
  • Add validation of keystore setting names #27626
Snapshot/Restore
  • Use AmazonS3.doesObjectExist() method in S3BlobContainer #27723
  • Remove XContentType auto detection in BlobStoreRepository #27480
  • Include include_global_state in Snapshot status API (#22423) #26853 (issue: #22423)
Task Manager
  • Add ability to associate an ID with tasks #27764 (issue: #23250)
Translog

Bug Fixes

edit
Aggregations
  • Adds metadata to rewritten aggregations #28185 (issue: #28170)
  • Fix NPE on composite aggregation with sub-aggregations that need scores #28129
  • StringTerms.Bucket.getKeyAsNumber detection type #28118 (issue: #28012)
  • Fix incorrect results for aggregations nested under a nested aggregation #27946 (issue: #27912)
  • Fix global aggregation that requires breadth first and scores #27942 (issues: #22321, #27928)
  • Fix composite aggregation when after term is missing in the shard #27936
  • Fix preserving FiltersAggregationBuilder#keyed field on rewrite #27900 (issue: #27841)
  • Using DocValueFormat::parseBytesRef for parsing missing value parameter #27855 (issue: #27788)
  • Fix illegal cast of the "low cardinality" optimization of the terms aggregation. #27543
  • Always include the _index and _id for nested search hits. #27201 (issue: #27053)
Allocation
  • Do not open indices with broken settings #26995
Core
  • Fix lock accounting in releasable lock #28202
  • Fixes ByteSizeValue to serialise correctly #27702 (issue: #27568)
  • Do not set data paths on no local storage required #27587 (issue: #27572)
  • Ensure threadcontext is preserved when refresh listeners are invoked #27565
  • Ensure logging is configured for CLI commands #27523 (issue: #27521)
Engine
  • Replica recovery could go into an endless flushing loop #28350
  • Use _refresh to shrink the version map on inactivity #27918 (issue: #27852)
  • Allow resize version map under lock even if there are pending operations #27870 (issue: #27852)
  • Reset LiveVersionMap on sync commit #27534 (issue: #27516)
Geo
  • Correct two equality checks on incomparable types #27688
  • Handle case where the hole vertex is south of the containing polygon(s) #27685 (issue: #25933)
Highlighting
  • Fix highlighting on a keyword field that defines a normalizer #27604
Inner Hits
  • Add version support for inner hits in field collapsing (#27822) #27833 (issue: #27822)
Internal
  • Never return null from Strings.tokenizeToStringArray #28224 (issue: #28213)
  • Fallback to TransportMasterNodeAction for cluster health retries #28195 (issue: #28169)
  • Retain originalIndex info when rewriting FieldCapabilities requests #27761
Java REST Client
  • Do not use system properties when building the HttpAsyncClient #27829 (issue: #27827)
Mapping
  • Ignore null value for range field (#27845) #28116 (issue: #27845)
  • Pass java.locale.providers=COMPAT to Java 9 onwards #28080 (issue: #10984)
  • Allow update of eager_global_ordinals on _parent. #28014 (issue: #24407)
  • Fix merging of _meta field #27352 (issue: #27323)
Network
  • Only bind loopback addresses when binding to local #28029 (issue: #1877)
  • Remove potential nio selector leak #27825
  • Fix issue where the incorrect buffers are written #27695 (issue: #27551)
  • Throw UOE from compressible bytes stream reset #27564 (issue: #24927)
  • Bubble exceptions when closing compressible streams #27542 (issue: #27540)
Packaging
  • Allow custom service names when installing on windows #25255 (issue: #25231)
Percolator
  • Avoid TooManyClauses exception if number of terms / ranges is exactly equal to 1024 #27519 (issue: #1)
Plugin Analysis ICU
  • Catch InvalidPathException in IcuCollationTokenFilterFactory #27202
Plugin Analysis Phonetic
  • Fix daitch_mokotoff phonetic filter to use the dedicated Lucene filter #28225 (issue: #28211)
Plugin Lang Painless
  • Painless: Fix variable scoping issue in lambdas #27571 (issue: #26760)
  • Painless: Fix errors allowing void to be assigned to def. #27460 (issue: #27210)
Plugin Repository HDFS
  • Fix SecurityException when HDFS Repository used against HA Namenodes #27196
Plugins
  • Make sure that we don’t detect files as maven coordinate when installing a plugin #28163
  • Fix upgrading indices which use a custom similarity plugin. #26985 (issue: #25350)
Recovery
Rollover
Scripting
  • Ensure we protect Collections obtained from scripts from self-referencing #28335
Scroll
  • Reject scroll query if size is 0 (#22552) #27842 (issue: #22552)
  • Fix scroll query with a sort that is a prefix of the index sort #27498
Search
  • Fix simple_query_string on invalid input #28219 (issue: #28204)
  • Use the underlying connection version for CCS connections #28093
  • Fix synonym phrase query expansion for cross_fields parsing #28045
  • Carry forward weights, etc on rescore rewrite #27981 (issue: #27979)
  • Fix routing with leading or trailing whitespace #27712 (issue: #27708)
Sequence IDs
  • Recovery from snapshot may leave seq# gaps #27850
  • No longer unidle shard during recovery #27757 (issue: #26591)
  • Obey translog durability in global checkpoint sync #27641
Settings
  • Settings: Introduce settings updater for a list of settings #28338 (issue: #28047)
  • Fix setting notification for complex setting (affixMap settings) that could cause transient settings to be ignored #28317 (issue: #28316)
  • Fix environment variable substitutions in list setting #28106 (issue: #27926)
  • Allow index settings to be reset by wildcards #27671 (issue: #27537)
Snapshot/Restore
  • Consistent updates of IndexShardSnapshotStatus #28130 (issue: #26480)
  • Avoid concurrent snapshot finalizations when deleting an INIT snapshot #28078 (issues: #27214, #27931, #27974)
  • Do not start snapshots that are deleted during initialization #27931
  • Do not swallow exception in ChecksumBlobStoreFormat.writeAtomic() #27597
  • Consistent update of stage and failure message in IndexShardSnapshotStatus #27557 (issue: #26480)
  • Fail restore when the shard allocations max retries count is reached #27493 (issue: #26865)
  • Delete shard store files before restoring a snapshot #27476 (issues: #20220, #26865)
Stats
  • Fixes DocStats to properly deal with shards that report -1 index size #27863
  • Include internal refreshes in refresh stats #27615
Term Vectors
  • Fix term vectors generator with keyword and normalizer #27608 (issue: #27320)

Upgrades

edit
Core
  • Dependencies: Update joda time to 2.9.9 #28261
  • upgrade to lucene 7.2.1 #28218 (issue: #28044)
  • Upgrade jna from 4.4.0-1 to 4.5.1 #28183 (issue: #28172)
Ingest
  • update ingest-attachment to use Tika 1.17 and newer deps #27824