CSV processor

edit

Extracts fields from CSV line out of a single text field within a document. Any empty field in CSV will be skipped.

Table 37. CSV Options

Name Required Default Description

field

yes

-

The field to extract data from

target_fields

yes

-

The array of fields to assign extracted values to

separator

no

,

Separator used in CSV, has to be single character string

quote

no

"

Quote used in CSV, has to be single character string

ignore_missing

no

true

If true and field does not exist, the processor quietly exits without modifying the document

trim

no

false

Trim whitespaces in unquoted fields

empty_value

no

-

Value used to fill empty fields, empty fields will be skipped if this is not provided. Empty field is one with no value (2 consecutive separators) or empty quotes ("")

if

no

-

Conditionally execute this processor.

on_failure

no

-

Handle failures for this processor. See Handling Failures in Pipelines.

ignore_failure

no

false

Ignore failures for this processor. See Handling Failures in Pipelines.

tag

no

-

An identifier for this processor. Useful for debugging and metrics.

{
  "csv": {
    "field": "my_field",
    "target_fields": ["field1", "field2"]
  }
}

If the trim option is enabled then any whitespace in the beginning and in the end of each unquoted field will be trimmed. For example with configuration above, a value of A, B will result in field field2 having value {nbsp}B (with space at the beginning). If trim is enabled A, B will result in field field2 having value B (no whitespace). Quoted fields will be left untouched.