Known limitations

Running as root and within a single namespace (ECK < 2.10.0 and Agent < 7.14.0)

Until version 7.14.0 and ECK version 2.10.0, Elastic Agent in Fleet mode has to run as root and in the same namespace as the Elasticsearch cluster it connects to.

This was due to configuration limitations in Fleet/Elastic Agent. ECK needed to establish trust between Elastic Agents and Elasticsearch. ECK was only able to fetch the required Elasticsearch CA correctly if both resources are in the same namespace. As of Elastic Stack version 7.14.0 and ECK version 2.10.0 it is also possible to run Elastic Agent and Fleet as a non-root user. See Storing local state in host path volume for instructions. To establish trust, the Pod needs to update the CA store through a call to update-ca-trust before Elastic Agent runs. To call it successfully, the Pod needs to run with elevated privileges.

Running Endpoint Security integration

edit

Running Endpoint Security integration is not yet supported in containerized environments, like Kubernetes. This is not an ECK limitation, but the limitation of the integration itself. Note that you can use ECK to deploy Elasticsearch, Kibana and Fleet Server, and add Endpoint Security integration to your policies if Elastic Agents running those policies are deployed in non-containerized environments.

Fleet Server initialization fails on minikube when CNI is disabled

edit

When deployed with ECK, the Fleet Server Pod makes an HTTP call to itself during Fleet initialization using its Service. Since a Pod cannot reach itself through its Service on minikube when CNI is disabled, the call hangs until the connection times out and the Pod enters a crash loop.

Solution: enable CNI when starting minikube: minikube start --cni=true.

Storing local state in host path volume

edit

Elastic Agent managed by ECK stores local state in a host path volume by default. This ensures that integrations run by the agent can continue their work without duplicating work that has already been done after the Pod has been recreated for example because of a Pod configuration change. Multiple replicas of an agent, for example Fleet Servers, can not be deployed on the same underlying Kubernetes node as they would try to use the same host path. There are 2 options for managing this feature:

If local state storage in hostPath volumes is not desired this can be turned off by configuring an emptyDir volume instead.
If local state storage is still desired but running the Agent container as root is not allowed, then you can run a DaemonSet that adjusts the permissions for the Agent local state on each Node prior to running Elastic Agent. Note that this DaemonSet must be runAsUser: 0 and possibly privileged: true. Also note the Kibana changes required to trust the Elasticsearch CA when running in fleet mode.

Full configuration examples exist in Running as a non-root user.

« Configuration Examples Run Elastic Maps Server on ECK »