WARNING: Version 6.1 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Lowercase Tokenizer UAX URL Email Tokenizer »
Elastic Docs ›Elasticsearch Reference [6.1] ›Analysis ›Tokenizers

Whitespace Tokenizer

edit

Whitespace Tokenizer

edit

The whitespace tokenizer breaks text into terms whenever it encounters a whitespace character.

Example output

edit
POST _analyze
{
  "tokenizer": "whitespace",
  "text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}

The above sentence would produce the following terms:

[ The, 2, QUICK, Brown-Foxes, jumped, over, the, lazy, dog's, bone. ]

Configuration

edit

The whitespace tokenizer accepts the following parameters:

max_token_length

The maximum token length. If a token is seen that exceeds this length then it is split at max_token_length intervals. Defaults to 255.

« Lowercase Tokenizer UAX URL Email Tokenizer »