It is time to say goodbye: This version of Elastic Cloud Enterprise has reached end-of-life (EOL) and is no longer supported.

The documentation for this version is no longer being maintained. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Cross-cluster search profile Customize your deployment »
Elastic Docs ›Elastic Cloud Enterprise Reference [2.10] ›Getting started with deployments ›Create your deployment ›What are hardware profiles?

Hot-warm architecture profile

edit

Hot-warm architecture profile

edit

Starting in Elastic stack version 7.10, all deployments on Elastic Cloud Enterprise support hot, warm, and cold data tiers. You no longer need to use a dedicated hot-warm architecture profile to manage your time series data, so this profile type is no longer available.

A profile that you typically use for time-series analytics and log aggregation workloads that benefit from tiered-storage automatic index curation. Includes features to manage resources efficiently when you need greater capacity, such as:

  • A tiered architecture with two different types of data nodes, hot and warm.
  • Time-based indices, with automatic index curation to move indices from hot to warm nodes over time by changing their shard allocation.

The two type of data nodes in a hot-warm architecture each have their own characteristics:

Hot data node
Handles all indexing of new data in the cluster and holds the most recent daily indices that tend to be queried most frequently. Indexing is an I/O intensive activity and the hardware these nodes run on needs to be more powerful and use SSD storage.
Warm data node
Handles a large amount of read-only indices that are not queried frequently. With read-only indices, warm nodes can use very large spindle drives instead of SSD storage. Reducing the overall cost of retaining data over time yet making it accessible for queries.

Index curation

edit

One of the key features of a hot-warm architecture, time-based index curation automates the task of moving data from hot to warm nodes as it ages. When you deploy a hot-warm architecture, Elastic Cloud Enterprise performs regular index curation according to these rules:

  • Index curation moves indices from one Elasticsearch node to another by changing their shard allocation, always from hot to warm.
  • Index curation is always time-based and takes place when an index reaches the age specified, in days, weeks, or months.
  • Index curation always targets indexes according to one or more matching patterns. If an index matches a pattern, Elastic Cloud Enterprise moves it from a hot to a warm node.

While you create your deployment, you can define which indices get curated and when. To know more about index curation, see Configure index management

To know more about how hot-warm architectures work with Elasticsearch, see “Hot-Warm” Architecture in Elasticsearch 5.x.

In this profile
edit

The following features are included with this profile:

  • Elasticsearch:

    • Data nodes - hot: Starts at 4 GB memory x 1 availability zone. Uses the data.default instance configuration.
    • Data nodes - warm: Starts at 4 GB memory x 1 availability zone. Uses the data.highstorage instance configuration.

    • Master nodes:

      Additional master-eligible node added when choosing 2 availability zones (to create a quorum of 3).

      When 1 AZ or 3 AZ are selected, the data nodes act as master-eligible node and there is no requirement for an additional master-eligible node.

      Configurations beyond 5 nodes per AZ can also spin up a dedicated master-eligible set of nodes (in 3 AZs always) to offload the data nodes. Uses the master instance configuration.

  • Kibana: Starts at 1 GB memory x 1 availability zone. Uses the kibana instance configuration.
  • Machine learning (ML): Disabled by default. The functionality is pre-wired into the profile, but you must explicitly enable it in the UI. Uses the ml instance configuration.
  • APM (application performance monitoring): Disabled by default. The functionality is pre-wired into the profile, but you must explicitly enable it in the UI. Uses the apm instance configuration.

To use this profile effectively, you must tag your allocators and edit the default instance configurations, so that ECE knows where to host the Elastic Stack products that are part of your deployment.

« Cross-cluster search profile Customize your deployment »