What’s new in 8.11

Here are the highlights of what’s new and improved in 8.11. For detailed information about this release, check the release notes.

Previous versions: 8.10 | 8.9 | 8.8 | 8.7 | 8.6 | 8.5 | 8.4 | 8.3 | 8.2 | 8.1 | 8.0

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. In 8.11 we are introducing Elasticsearch Query Language (ES|QL), Elastic’s new piped language for data exploration and investigation. ES|QL transforms, enriches, and simplifies your data exploration process.

Here is what you can expect:

ES|QL in Discover brings efficiency and power to your data investigations, streamlining your path to insights.

Learn more about ES|QL’s capabilities in the Elasticsearch ES|QL documentation.

You can now edit a Lens visualization without leaving the dashboard instead of navigating back and forth to the Lens editor. Open the panel menu and select Edit visualization. A flyout will be open in the dashboard where you can perform any edits to your Lens panels. Once happy with your edits, click Apply and close. This new editing experience is more convenient and will save you time since the dashboard will not need to reload when saving your changes.

You can now easily navigate from one dashboard to another using the links panel. Better organize your dashboards and make them more performant by chunking them in multiple dashboards with fewer visualizations and linking them together. You can carry over your filters, query, and time range when navigating to other related dashboards. Display your links horizontally or vertically as it better suits your dashboard layout. You can also use the links panel to include external links in your dashboards, such as to your wiki page or other applications. Decide whether you want to open the links in the same browser tab or in a new one.

Color is a fundamental visual element, alongside position and shape, that plays a crucial role in conveying information effectively.

Here’s what our new color mapping feature offers:

Whether you’re working with cartesian, partition, or tag clouds charts, these enhancements are designed to help you make the most of your data. With this feature, you can expect an improved ability to categorize, differentiate, and emphasize data points on your charts, ultimately leading to better insights and more visually appealing dashboards.

Customers querying data from multiple clusters (CCS queries) will get more information about each of the cluster’s responses. For each of the visualizations in a dashboard as well as in Discover, you can look at response times per cluster and shard failures. This is especially important when one or more clusters are not able to provide all the data, so you know that you are looking at partial results and why they fail.

We’ve introduced full annotation group editing in the Visualize Library. Now, you can easily edit shared annotation groups without leaving Lens. No more searching for consuming visualizations.

In 8.8, we introduced Elastic Learned Sparse Encoder in technical preview. ELSER is Elastic’s text expansion language model for AI search. It offers superior relevance out of the box, without the need for retraining on in-domain data or any other ML or MLOps effort. Deploy it with a couple of clicks from Elastic’s UI and start leveraging the power of AI with your search.

In 8.11, we’re releasing a generally available second version. ELSER model-2 comes in two versions:

Both ELSER model-2 versions, platform-optimized and cross-platform, show improved relevance compared to the original ELSER release, as measured against the BEIR benchmarks. Importantly, the optimized version also shows significantly improved performance (reduced inference latency). The Elastic Cloud supports the optimized version and so Elastic Cloud users will benefit from the materially improved performance of the optimized ELSER model-2.

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. We are working to introduce a unified inference API that abstracts away the complexity of performing inference on different models that are trained for different tasks. The API introduces a simple, intuitive syntax of the form:

POST /_inference/<task_type>/<model_id>

In 8.11, we’re’releasing a contained first MVP iteration of this framework, which initially only supports ELSER. This greatly simplifies the syntax for creating an inference pipeline.

More importantly, in the future the new inference API will support both internal and external models and will integrate with the LLM ecosystem for our users to have the most powerful AI effortlessly and seamlessly at their fingertips, through a unified, self-explanatory API.

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Continuing enhancing log analysis capabilities with smart AIOps tools for drastically shorter mean time to repair. We now support detection of log rate changes that are due to text fields, for example the common message log field. Previously log rate change was limited to detecting spikes and dips caused by keyword fields. By adding text fields, we’ve incorporated pattern analysis into log rate analysis. That has significantly reduced the time it takes to detect and diagnose events that used to go unnoticed for periods of time.

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. In 8.10, we introduced the Data comparison view to help you detect data drift. In 8.11, we have renamed it to Data drift and we have enhanced it to include workflows that help you visualize changes in the model input data and detect potential model performance degradation over time.

We have improved the UX for attaching anomaly swim lanes and anomaly charts to dashboards. It is now more friendly and consistent with the same functionality from Lens. In addition, you can now attach these ML charts to new dashboards (previously this was only able to be done for existing ones).

In Data Frame Analytics, we previously added the ability to link directly to Discover and Dashboards from the results data grid filtering for the row’s field/values for all visible columns. For improved usability, you can now do this during the job creation as well. We have also made UX improvements for the deployment of trained models from Data Frame Analytics jobs, including an option to reindex your data at the end of the ingest pipeline creation.

A new ES|QL alerting rule type is now available under the existing Elasticsearch rule type. This rule type brings all the new functionalities that are available within the new and powerful language, ES|QL, to Kibana Alerting to allow and unlock new alerting use cases.

With the new type, users will be able to generate a single alert based on defined ES|QL query and preview the query result before saving the rule. When the query returns an empty result no alerts will be generated.

A new functionality is now available in Kibana cases. Users will be able to add custom fields to the case structure, so they can use it for better classification and case enrichment. As a first step, those fields will be available in the case view only. In the next step, we’re planning to have more field types, dedicated privileges, support filters, and search capabilities in the case table.

The existing Elasticsearch alerting rule is now supported by multiple selection when grouping by alert fields, which allows you to define multiple layers of term aggregations.

The Slack connector supports a new allow list, so customers will be able to manage the available Slack channels within the alert actions.

The Elasticsearch alerting rule is now available in Observability. Before this update, Observability customers who leverage the Elasticsearch rule were required to move between Stack Management and Observability to manage their alerts. To manage the Observability roles properly, we added a new field to determine which role can maintain the created rule and its generated alerts.

In support of our new ES|QL capabilities, we have added enrich policies to our Index Management experience. Users can now create their enrichment policies right from here and get started using it right away. Once configured, all enrich policies are available in the Enrich Policies tab.

Example of an ES|QL Query using an enrich policy with enrich:

Example of the enrich policies shown in the UI:

Discover searches are powered by KQL by default. You can easily switch to ES|QL mode within Discover and search your data with this new query language. You can also access ES|QL in Discover from the global search bar within Elastic. With just a few keystrokes, simply type in “ESQL” and you can access Discover with ES|QL enabled for you.

You can now view your Elasticsearch endpoint, Cloud ID, and even manage your API keys from many areas in Elastic such as integrations. You can also access the connection details on any page in Elastic using the help menu from the header bar.

When subscribing to Elastic using the AWS Marketplace, users have the option to quickly get set up with an AWS CloudFormation Template. This step allows you to create an Elastic deployment in the AWS region of your choice. We’ve updated the AWS CloudFormation Template with bug fixes and stability improvements to better help you get started.