Elastic Managed LLMs
Serverless Stack
Elastic provides built-in LLMs through the Elastic Managed LLM connector vetted for GenAI product features across our platform. Currently, the following built-in LLMs are available:
Serverless Stack
A high-performance, general-purpose language model suitable for the widest range of use cases. Model details are available in the model card.
Serverless Stack
General Purpose LLM v1 is the default LLM for the Elastic Managed LLM connector.
- The default LLM may change in the future based on evaluations of performance, security, and accuracy.
- This LLM was called Elastic Managed LLM before 9.3.
Details of the currently used model are available in the model card.
- Requires the
manage_inferencecluster privilege (the built-ininference_adminrole grants this privilege)
The Elastic Managed LLMs use a third party service provider for inference. Currently, this is AWS Bedrock in AWS US regions, beginning with us-east-1. Additional providers may be added in future.
Customer projects or deployments hosted in any cloud service provider or region have access to the Elastic Managed LLMs in the AWS US region us-east-1.
All data is encrypted in transit. The LLMs are configured for zero data retention: none of the prompts or outputs are stored by the service provider.
Only request metadata is logged in AWS CloudWatch. No information related to prompts is retained. Logged metadata includes the timestamp, model used, region, and request status.
Read more at our AI Data FAQs to learn about our data practices for AI related features.
The Elastic Managed LLMs incur a cost per million tokens for input and output tokens. Refer to the Elastic Cloud pricing pages for details.