Elasticsearch for Apache Hadoop 6.7.0 released

We’re excited to announce the release of Elasticsearch for Apache Hadoop (aka ES-Hadoop) 6.7.0 built against Elasticsearch 6.7.0.

Getting ready for 7.0

Cascading is now deprecated

We've seen the usage statistics dropping for the Cascading integration over the last couple of years. These include download numbers, community posts, and open issues. Additionally, ES-Hadoop only supports the Cascading 2 line. Cascading is already well into it's 3.X line but we haven't seen much interest or pressure to get on to any of the newer versions.

Each integration that we support incurs a cost for testing and maintenance as new features are added. Unfortunately, we feel that the Cascading integration is not resulting in enough benefit for our entire user base, and the effort to maintain it could be better spent on features and fixes in other areas.

Due to these factors, the Cascading integration has been deprecated, and will be removed in the 7.0.0 release.

What’s new?

Support for Elasticsearch Kerberos authentication

Beginning in 6.7.0, Elasticsearch for Apache Hadoop now supports using Kerberos to authenticate to Elasticsearch. If you have a cluster that is secured with Kerberos, you can configure Elasticsearch security to use a Kerberos based realm. When launching jobs that run on that cluster, any Kerberos credentials that are available to the job can be picked up and used to authenticate client connections to Elasticsearch. The authentication mechanisms are available to all integrations, with some including additional support for managing credentials on long lived Storm topologies and Spark Streaming jobs. Take a look at the new documentation for configuring Kerberos for ES-Hadoop and tell us what you think!

We love hearing your feedback and suggestions. If you have a great idea for a new feature or enhancement, or if you have any questions, stop by our forums or submit an issue on Github!