- .NET Clients: other versions:
- Introduction
- Installation
- Breaking changes
- API Conventions
- Elasticsearch.Net - Low level client
- NEST - High level client
- Troubleshooting
- Search
- Query DSL
- Full text queries
- Term level queries
- Exists Query Usage
- Fuzzy Date Query Usage
- Fuzzy Numeric Query Usage
- Fuzzy Query Usage
- Ids Query Usage
- Prefix Query Usage
- Date Range Query Usage
- Long Range Query Usage
- Numeric Range Query Usage
- Term Range Query Usage
- Regexp Query Usage
- Term Query Usage
- Terms Set Query Usage
- Terms List Query Usage
- Terms Lookup Query Usage
- Terms Query Usage
- Wildcard Query Usage
- Compound queries
- Joining queries
- Geo queries
- Specialized queries
- Span queries
- NEST specific queries
- Aggregations
- Metric Aggregations
- Average Aggregation Usage
- Boxplot Aggregation Usage
- Cardinality Aggregation Usage
- Extended Stats Aggregation Usage
- Geo Bounds Aggregation Usage
- Geo Centroid Aggregation Usage
- Geo Line Aggregation Usage
- Max Aggregation Usage
- Median Absolute Deviation Aggregation Usage
- Min Aggregation Usage
- Percentile Ranks Aggregation Usage
- Percentiles Aggregation Usage
- Rate Aggregation Usage
- Scripted Metric Aggregation Usage
- Stats Aggregation Usage
- String Stats Aggregation Usage
- Sum Aggregation Usage
- T Test Aggregation Usage
- Top Hits Aggregation Usage
- Top Metrics Aggregation Usage
- Value Count Aggregation Usage
- Weighted Average Aggregation Usage
- Bucket Aggregations
- Adjacency Matrix Usage
- Auto Date Histogram Aggregation Usage
- Children Aggregation Usage
- Composite Aggregation Usage
- Date Histogram Aggregation Usage
- Date Range Aggregation Usage
- Diversified Sampler Aggregation Usage
- Filter Aggregation Usage
- Filters Aggregation Usage
- Geo Distance Aggregation Usage
- Geo Hash Grid Aggregation Usage
- Geo Tile Grid Aggregation Usage
- Global Aggregation Usage
- Histogram Aggregation Usage
- Ip Range Aggregation Usage
- Missing Aggregation Usage
- Multi Terms Aggregation Usage
- Nested Aggregation Usage
- Parent Aggregation Usage
- Range Aggregation Usage
- Rare Terms Aggregation Usage
- Reverse Nested Aggregation Usage
- Sampler Aggregation Usage
- Significant Terms Aggregation Usage
- Significant Text Aggregation Usage
- Terms Aggregation Usage
- Variable Width Histogram Usage
- Pipeline Aggregations
- Average Bucket Aggregation Usage
- Bucket Script Aggregation Usage
- Bucket Selector Aggregation Usage
- Bucket Sort Aggregation Usage
- Cumulative Cardinality Aggregation Usage
- Cumulative Sum Aggregation Usage
- Derivative Aggregation Usage
- Extended Stats Bucket Aggregation Usage
- Max Bucket Aggregation Usage
- Min Bucket Aggregation Usage
- Moving Average Ewma Aggregation Usage
- Moving Average Holt Linear Aggregation Usage
- Moving Average Holt Winters Aggregation Usage
- Moving Average Linear Aggregation Usage
- Moving Average Simple Aggregation Usage
- Moving Function Aggregation Usage
- Moving Percentiles Aggregation Usage
- Normalize Aggregation Usage
- Percentiles Bucket Aggregation Usage
- Serial Differencing Aggregation Usage
- Stats Bucket Aggregation Usage
- Sum Bucket Aggregation Usage
- Matrix Aggregations
- Metric Aggregations
Ingest Pipelines
editIngest Pipelines
editAn ingest pipeline is a series of processors that are to be executed in the same order as they are declared.
Let’s work with the following POCOs
public class Person { public int Id { get; set; } public string FirstName { get; set; } public string LastName { get; set; } public string IpAddress { get; set; } public GeoIp GeoIp { get; set; } } public class GeoIp { public string CityName { get; set; } public string ContinentName { get; set; } public string CountryIsoCode { get; set; } public GeoLocation Location { get; set; } public string RegionName { get; set; } }
Create an ingestion pipeline
editAssuming we are indexing Person documents, we can create an ingestion pipeline that manipulates the incoming values before they are indexed.
Lets assume that our application always expects surnames to be capitalised, and for initials to be indexed into their own field. We also have an IP address that we’d like to convert into a human-readable location.
We could achieve this requirement by creating a custom mapping and creating an ingest pipeline. The Person type can then be used as-is, without making any changes.
client.Indices.Create("people", c => c .Map<Person>(p => p .AutoMap() .Properties(props => props .Keyword(t => t.Name("initials")) .Ip(t => t.Name(dv => dv.IpAddress)) .Object<GeoIp>(t => t.Name(dv => dv.GeoIp)) ) ) ); client.Ingest.PutPipeline("person-pipeline", p => p .Processors(ps => ps .Uppercase<Person>(s => s .Field(t => t.LastName) ) .Script(s => s .Lang("painless") .Source("ctx.initials = ctx.firstName.substring(0,1) + ctx.lastName.substring(0,1)") ) .GeoIp<Person>(s => s .Field(i => i.IpAddress) .TargetField(i => i.GeoIp) ) ) ); var person = new Person { Id = 1, FirstName = "Martijn", LastName = "Laarman", IpAddress = "139.130.4.5" }; var indexResponse = client.Index(person, p => p.Index("people").Pipeline("person-pipeline"));
automatically create the mapping from the type |
|
create an additional field to store the initials |
|
map field as IP Address type |
|
map GeoIp as object |
|
uppercase the lastname |
|
use a painless script to populate the new field |
|
use ingest-geoip plugin to enrich the GeoIp object from the supplied IP Address |
|
index the document using the created pipeline |
Increasing timeouts
editWhen a pipeline is specified, there will be the added overhead of document enrichment when indexing, the example given above, the execution of the uppercasing and the Painless script.
For large bulk requests, it could be prudent to increase the default indexing timeout to avoid exceptions.
On this page