IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› › ›

Update transform API

edit

Update transform API

edit

Updates certain properties of a transform.

Request

edit

POST _transform/<transform_id>/_update

Prerequisites

edit

If the Elasticsearch security features are enabled, you must have the following built-in roles and privileges:

transform_admin
manage_transform (the built-in transform_admin role has this privilege)
source index: read, view_index_metadata
destination index: read, create_index, index

For more information, see Security privileges and Built-in roles.

Description

edit

This API updates an existing transform. The list of properties that you can update is a subset of the list that you can define when you create a transform.

When the transform is updated, a series of validations occur to ensure its success. You can use the defer_validation parameter to skip these checks.

All updated properties except description do not take effect until after the transform starts the next checkpoint. This is so there is consistency with the pivoted data in each checkpoint.

When Elasticsearch security features are enabled, your transform remembers which roles the user who updated it had at the time of update and runs with those privileges.
You must use Kibana or this API to update a transform. Do not update a transform directly via .transform-internal* indices using the Elasticsearch index API. If Elasticsearch security features are enabled, do not give users any privileges on .transform-internal* indices. If you used transforms prior 7.5, also do not give users any privileges on .data-frame-internal* indices.

Path parameters

edit

<transform_id>: (Required, string) Identifier for the transform. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

edit

defer_validation: (Optional, Boolean) When true, deferrable validations are not run. This behavior may be desired if the source index does not exist until after the transform is updated.

Request body

edit

description: (Optional, string) Free text description of the transform.

dest

(Optional, object) The destination for the transform.

Properties of dest

index: (Required, string) The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the Create index API prior to starting the transform.
pipeline: (Optional, string) The unique identifier for an ingest pipeline.

frequency

(Optional, time units) The interval between checks for changes in the source indices when the transform is running continuously. Also determines the retry interval in the event of transient failures while the transform is searching or indexing. The minimum value is 1s and the maximum is 1h. The default value is 1m.

settings

(Optional, object) Defines optional transform settings.

Properties of settings

dates_as_epoch_millis: (Optional, boolean) Defines if dates in the ouput should be written as ISO formatted string (default) or as millis since epoch. epoch_millis has been the default for transforms created before version 7.11. For compatible output set this to true. The default value is false.
docs_per_second: (Optional, float) Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.
max_page_search_size: (Optional, integer) Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 10,000. The default value is 500.

source

(Optional, object) The source of the data for the transform.

Properties of source

index: (Required, string or array) The source indices for the transform. It can be a single index, an index pattern (for example, "my-index-*"), an array of indices (for example, ["my-index-000001", "my-index-000002"]), or an array of index patterns (for example, ["my-index-*", "my-other-index-*"].
query: (Optional, object) A query clause that retrieves a subset of data from the source index. See Query DSL.

sync

(Optional, object) Defines the properties transforms require to run continuously.

Properties of sync

time

(Required, object) Specifies that the transform uses a time field to synchronize the source and destination indices.

Properties of time

delay: (Optional, time units) The time delay between the current time and the latest input data time. The default value is 60s.
field: (Required, string) The date field that is used to identify new documents in the source.

In general, it’s a good idea to use a field that contains the ingest timestamp. If you use a different field, you might need to set the delay such that it accounts for data transmission delays.

Examples

edit

POST _transform/simple-kibana-ecomm-pivot/_update
{
  "source": {
    "index": "kibana_sample_data_ecommerce",
    "query": {
      "term": {
        "geoip.continent_name": {
          "value": "Asia"
        }
      }
    }
  },
  "description": "Maximum priced ecommerce data by customer_id in Asia",
  "dest": {
    "index": "kibana_sample_data_ecommerce_transform_v2",
    "pipeline": "add_timestamp_pipeline"
  },
  "frequency": "15m",
  "sync": {
    "time": {
      "field": "order_date",
      "delay": "120s"
    }
  }
}

When the transform is updated, you receive the updated configuration:

{
  "id": "simple-kibana-ecomm-pivot",
  "source": {
    "index": ["kibana_sample_data_ecommerce"],
    "query": {
      "term": {
        "geoip.continent_name": {
          "value": "Asia"
        }
      }
    }
  },
  "pivot": {
    "group_by": {
      "customer_id": {
        "terms": {
          "field": "customer_id"
        }
      }
    },
    "aggregations": {
      "max_price": {
        "max": {
          "field": "taxful_total_price"
        }
      }
    }
  },
  "description": "Maximum priced ecommerce data by customer_id in Asia",
  "dest": {
    "index": "kibana_sample_data_ecommerce_transform_v2",
    "pipeline": "add_timestamp_pipeline"
  },
  "frequency": "15m",
  "sync": {
    "time": {
      "field": "order_date",
      "delay": "120s"
    }
  },
  "settings": { },
  "version": "7.5.0",
  "create_time": 1518808660505
}

« Stop transforms API Usage API »