It is time to say goodbye: This version of Elastic Cloud Enterprise has reached end-of-life (EOL) and is no longer supported.

The documentation for this version is no longer being maintained. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Add plugins Advanced cluster configuration »
Elastic Docs ›Elastic Cloud Enterprise Reference [2.12] ›Administering deployments ›Change your deployment configuration

Add custom bundles and plugins to your deployment

edit

Add custom bundles and plugins to your deployment

edit

Follow these steps to upload custom bundles and plugins to your Elasticsearch clusters, so that it uses your custom bundles or plugins.

  • Update your Elasticsearch cluster in the advanced configuration editor:
  • For bundles, modify the resources.elasticsearch.plan.cluster_topology.elasticsearch.user_bundles JSON attribute of each Elasticsearch instance type.
  • For plugins, modify the resources.elasticsearch.plan.cluster_topology.elasticsearch.user_plugins JSON attribute of each Elasticsearch instance type.

We’ve provided some examples, including LDAP bundles, SAML bundles, and adding a JVM trustore.

As part of the ECE high availability strategy, it’s a good idea to make sure that any web servers hosting custom bundles or plugins required by ECE are available to all allocators, so that they can continue to be accessed in the event of a network partition or zone outage. An instance that requires custom bundles or plugins will be unable to start in the event that it can’t access the plugin web server.

Add custom plugins to your deployment

edit

Custom plugins can include the official Elasticsearch plugins not provided with Elastic Cloud Enterprise, any of the community-sourced plugins, or plugins that you write yourself.

  1. Log into the Cloud UI.

  2. From the Deployments page, select your deployment.

    Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.

  3. From your deployment menu, select Edit then go to the bottom of the page and click advanced Elasticsearch configuration.

  4. Within the plan find the user_plugins section of each Elasticsearch instance type. If you don’t find it, you can add it within the Elasticsearch umbrella as shown in the following examples.

    {
...
  "resources": {
    "elasticsearch": [
	 ...
        "plan": {
	 	 ...
          "cluster_topology": [
			...
              "elasticsearch": {
				...
                "user_bundles": [
                {
                    ....
                } ] ,
              "user_plugins": [
                {
                  "url" : "<some static non_expirable url>", 
                  "name" : "plugin_name",
                  "elasticsearch_version" : "<es_version>" 
                },
                {
                  "url": "http://www.MYURL.com/my-custom-plugin.zip",
                  "name": "my-custom-plugin",
                  "elasticsearch_version": "7.13.1"
                }
              ]
}

    The URL for the plugin must be always available. Make sure you host the plugin artifacts internally in a highly available environment. The URL must use the scheme http or https

    The version must match exactly your Elasticsearch version, such as 6.4.2. Wildcards (*) are not allowed.

    If the plugin URL becomes unreachable (if the URL changes at remote end, or connectivity to the remote web server has issues) you might encounter boot loops.

Don’t use the same URL to serve newer versions of the plugin. This may result in different nodes of the same cluster running different plugin versions.

  1. Save your changes.
  2. To verify that all nodes have the plugins installed, use one of these commands: GET /_nodes/plugins?filter_path=nodes.*.plugins or GET _cat/plugins?v

Example: Custom LDAP bundle

edit

This example adds a custom LDAP bundle for deployment level role-based access control (RBAC). To set platform level RBAC, see Configure RBAC.

  1. Prepare a custom bundle as a ZIP file that contains your keystore file with the private key and certificate inside of a truststore folder in the same way that you would on Elastic Cloud. This bundle allows all Elasticsearch containers to access the same keystore file through your ssl.truststore settings.

  2. In the advanced configuration editor, update your new Elasticsearch cluster with the custom bundle you have just created. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example:

    {
...
  "resources": {
    "elasticsearch": [
	 ...
        "plan": {
	 	 ...
          "cluster_topology": [
			...
              "elasticsearch": {
				...
                "user_bundles": [
                {
                  "name": "ldap-cert",
                  "url": "http://www.MYURL.com/ldapcert.zip", 
                  "elasticsearch_version": "*"
                }
              ]
            }
            ...

    The URLs for the bundle ZIP files (ldapcert.zip) must be always available. Make sure you host the plugin artifacts internally in a highly available environment.

    If the bundle URL becomes unreachable (if the URL changes at remote end, or connectivity to the remote web server has issues) you might encounter boot loops.

    Don’t use the same URL to serve newer versions of the bundle. This may result in different nodes of the same cluster running different bundle versions.

  3. Custom bundles are unzipped in /app/config/BUNDLE_DIRECTORY_STRUCTURE, where BUNDLE_DIRECTORY_STRUCTURE is the directory structure within the bundle ZIP file itself. These file locations are needed in the next step.

    $ tree .
.
└── truststore
      └── keystore.ks

    In this example, the unzipped keystore file gets placed under /app/config/truststore/keystore.ks.

Example: Custom SAML bundle

edit

This example adds a custom SAML bundle for deployment level role-based access control (RBAC). To set platform level RBAC, see Configure RBAC.

  1. If your Identity Provider doesn’t publish its SAML metadata at an HTTP URL, or if your Elasticsearch cluster cannot reach that URL, you can upload the SAML metadata as a file.

    1. Prepare a ZIP file with a custom bundle that contains your Identity Provider’s metadata (metadata.xml) and store it in the saml folder.

      This bundle allows all Elasticsearch containers to access the metadata file.

    2. In the advanced configuration editor, update your Elasticsearch cluster configuration with the bundle you prepared in the previous step. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example:

      {
...
  "resources": {
    "elasticsearch": [
	 ...
        "plan": {
	 	 ...
          "cluster_topology": [
			...
              "elasticsearch": {
				...
                "user_bundles": [
                {
                  "name": "saml-metadata",
                  "url": "http://www.MYURL.com/saml-metadata.zip", 
                  "elasticsearch_version": "*"
                }
              ]
            }
            ...

      The URL for the bundle ZIP file must be always available. Make sure you host the plugin artifacts internally in a highly available environment.

      If the bundle URL becomes unreachable (if the URL changes at remote end, or connectivity to the remote web server has issues) you might encounter boot loops.

      Don’t use the same URL to serve newer versions of the bundle. This may result in different nodes of the same cluster running different bundle versions.

      Custom bundles are unzipped in /app/config/BUNDLE_DIRECTORY_STRUCTURE, where BUNDLE_DIRECTORY_STRUCTURE is the directory structure within the ZIP file itself. These file locations are needed in the next step.

      In this example, the SAML metadata file is located in the path /app/config/saml/metadata.xml:

      $ tree .
.
└── saml
      └── metadata.xml

    3. Adjust your saml realm configuration accordingly:

          idp.metadata.path: /app/config/saml/metadata.xml

      The path to the SAML metadata file that was uploaded

Example: Custom JVM trust store bundle

edit

This functionality is supported for Elasticsearch versions 6.x, starting with 6.8.2, and Elasticsearch 7.x, starting with 7.2.1

If you are using SSL certificates signed by non-public certificate authorities, Elasticsearch is not able to communicate with the services using those certificates unless you import a custom JVM trust store containing the certificates of your signing authority into your Elastic Cloud Enterprise installation. You’ll need the trust store to access snapshot repositories like Minio, for your Elastic Cloud Enterprise proxy, or to reindex from remote.

To import a JVM trust store:

  1. Prepare the custom JVM trust store:

    1. Pull the certificate from the service you want to make accessible:

        openssl s_client -connect <server using the certificate> -showcerts

      The server address (name and port number) of the service that you want Elasticsearch to be able to access. This command prints the entire certificate chain to stdout. You can choose a certificate at any level to be added to the trust store.
    2. Save it to a file with as a PEM extension.

    3. Locate your JRE’s default trust store, and copy it to the current directory:

        cp <default trust store location> cacerts

      Default JVM trust store is typically located in $JAVA_HOME/jre/libs/security/cacerts

      Default trust store contains certificates of many well known root authorities that are trusted by default. If you only want to include a limited list of CAs to trust, skip this step, and simply import specific certificates you want to trust into an empty store as shown next

    4. Use keytool command from your JRE to import certificate(s) into the keystore:

      $JAVA_HOME/bin/keytool -keystore cacerts -storepass changeit -noprompt -importcert -file <certificate>.pem -alias <some alias>

      The file where you saved the certificate to import, and an alias you assign to it, that is descriptive of the origin of the certificate

      We recommend that you keep file name and password for the trust store as JVM defaults (cacerts and changeit respectively). If you must use different values for these you will need to add extra configuration (see below) in addition to adding the bundle.

      You can have multiple certificates to the trust store, repeating the same command. There is only one trust store per cluster currently supported. You cannot, for example, add multiple bundles with different trust stores to the same cluster, they will not get merged. Add all certificates to be trusted to the same trust store

  2. Create the bundle:

     zip cacerts.zip cacerts

    The name of the zip archive is not significant

    A bundle may contain other contents beyond the trust store if you prefer, but we recommend creating separate bundles for different purposes.

  3. In the advanced configuration editor, update your Elasticsearch cluster configuration with the bundle you prepared in the previous step. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example:

    {
...
  "resources": {
    "elasticsearch": [
	 ...
        "plan": {
	 	 ...
          "cluster_topology": [
			...
              "elasticsearch": {
				...
                "user_bundles": [
                {
                  "name": "custom-ca-certs",
                  "url": "http://www.MYURL.com/cacerts.zip", 
                  "elasticsearch_version": "*" 
                }
              ]
            }
    ...

    The URL for the bundle ZIP file must be always available. Make sure you host the plugin artefacts internally in a highly available environment.

    Wildcards are allowed here, since the certificates are independent from the Elasticsearch version.

    If the bundle URL becomes unreachable (if the URL changes at remote end, or connectivity to the remote web server has issues) you might encounter boot loops.

Don’t use the same URL to serve newer versions of the bundle, i.e. when updating certificates. This may result in different nodes of the same cluster running different bundle versions.

  1. If you prefer to use a different file name and/or password for the trust store, you also need to add an additional configuration section to the cluster metadata before adding the bundle. This configuration should be added to the Elasticsearch cluster data section of the page:

      "trust": {
    "jvm_trust_store": "<filename included into bundle>", 
    "jvm_store_pass": "<password used to create keystore>" 
  }

The name of the trust store must match the filename included into the archive

Password used to create the trust store

Use only alphanumeric characters, dashes, and underscores in both file name and password.

You do not need to do this step if you are using default filename and password (cacerts and changeit respectively) in your bundle.

Example: Custom GeoIP database bundle

edit

  1. Prepare a ZIP file with a custom bundle that contains a: GeoLite2 database. The folder has to be named ingest-geoip, and the file name can be anything that is appended -(City|Country|ASN) with the mmdb file extension, and it must have a different name than the original name GeoLite2-City.mmdb.

    The file my-geoip-file.zip should look like this:

    $ tree .
.
└── ingest-geoip
    └── MyGeoLite2-City.mmdb
  2. Copy the ZIP file to a webserver that is reachable from any allocator in your environment.

  3. In the advanced configuration editor, update your Elasticsearch cluster configuration with the bundle you prepared in the previous step. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example.

    {
...
  "resources": {
    "elasticsearch": [
	 ...
        "plan": {
	 	 ...
          "cluster_topology": [
			...
              "elasticsearch": {
				...
                "user_bundles": [
                {
                  "name": "custom-geoip-db",
                  "url": "http://www.MYURL.com/my-geoip-file.zip",
                  "elasticsearch_version": "*"
                }
              ]
            }

  4. To use this bundle, you can refer it in the GeoIP processor of an ingest pipeline as MyGeoLite2-City.mmdb under database_file such as:

    ...
{
  "geoip": {
    "field": ...
    "database_file": "MyGeoLite2-City.mmdb",
    ...
  }
}
...
« Add plugins Advanced cluster configuration »