This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.

› › ›

Watsonx inference service

edit

Watsonx inference service

edit

Creates an inference endpoint to perform an inference task with the watsonxai service.

You need an IBM Cloud® Databases for Elasticsearch deployment to use the watsonxai inference service. You can provision one through the IBM catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, or Terraform.

Request

edit

PUT /_inference/<task_type>/<inference_id>

Path parameters

edit

<inference_id>

(Required, string) The unique identifier of the inference endpoint.

<task_type>

(Required, string) The type of the inference task that the model will perform.

Available task types:

text_embedding.

Request body

edit

service

(Required, string) The type of service supported for the specified task type. In this case, watsonxai.

service_settings

(Required, object) Settings used to install the inference model.

These settings are specific to the watsonxai service.

api_key

(Required, string) A valid API key of your Watsonx account. You can find your Watsonx API keys or you can create a new one on the API keys page.

You need to provide the API key only once, during the inference model creation. The Get inference API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

api_version

(Required, string) Version parameter that takes a version date in the format of YYYY-MM-DD. For the active version data parameters, refer to the documentation.

model_id

(Required, string) The name of the model to use for the inference task. Refer to the IBM Embedding Models section in the Watsonx documentation for the list of available text embedding models.

url

(Required, string) The URL endpoint to use for the requests.

project_id

(Required, string) The name of the project to use for the inference task.

rate_limit

(Optional, object) By default, the watsonxai service sets the number of requests allowed per minute to 120. This helps to minimize the number of rate limit errors returned from Watsonx. To modify this, set the requests_per_minute setting of this object in your service settings:

"rate_limit": {
    "requests_per_minute": <<number_of_requests>>
}

Watsonx AI service example

edit

The following example shows how to create an inference endpoint called watsonx-embeddings to perform a text_embedding task type.

resp = client.inference.put(
    task_type="text_embedding",
    inference_id="watsonx-embeddings",
    inference_config={
        "service": "watsonxai",
        "service_settings": {
            "api_key": "<api_key>",
            "url": "<url>",
            "model_id": "ibm/slate-30m-english-rtrvr",
            "project_id": "<project_id>",
            "api_version": "2024-03-14"
        }
    },
)
print(resp)

const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "watsonx-embeddings",
  inference_config: {
    service: "watsonxai",
    service_settings: {
      api_key: "<api_key>",
      url: "<url>",
      model_id: "ibm/slate-30m-english-rtrvr",
      project_id: "<project_id>",
      api_version: "2024-03-14",
    },
  },
});
console.log(response);

PUT _inference/text_embedding/watsonx-embeddings
{
    "service": "watsonxai",
    "service_settings": {
        "api_key": "<api_key>", 
        "url": "<url>", 
        "model_id": "ibm/slate-30m-english-rtrvr",
        "project_id": "<project_id>", 
        "api_version": "2024-03-14" 
    }
}

	A valid Watsonx API key. You can find on the API keys page of your account.
	The inference endpoint URL you created on Watsonx.
	The ID of your IBM Cloud project.
	A valid API version parameter. You can find the active version data parameters here.

« OpenAI inference service Info API »