This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.

« Search shard routing Search templates »

› › ›

Searching with query rules

edit

Searching with query rules

edit

Query rules allow customization of search results for queries that match specified criteria metadata. This allows for more control over results, for example ensuring that promoted documents that match defined criteria are returned at the top of the result list. Metadata is defined in the query rule, and is matched against the query criteria. Query rules use metadata to match a query. Metadata is provided as part of the search request as an object and can be anything that helps differentiate the query, for example:

A user-entered query string
Personalized metadata about users (e.g. country, language, etc)
A particular topic
A referring site
etc.

Query rules define a metadata key that will be used to match the metadata provided in the rule retriever with the criteria specified in the rule.

When a query rule matches the rule metadata according to its defined criteria, the query rule action is applied to the underlying organic query.

For example, a query rule could be defined to match a user-entered query string of pugs and a country us and promote adoptable shelter dogs if the rule query met both criteria.

Rules are defined using the query rules API and searched using the rule retriever or the rule query.

Rule definition

edit

When defining a rule, consider the following:

Rule type

edit

The type of rule we want to apply. We support the following rule types:

pinned will re-write the query into a pinned query, pinning specified results matching the query rule at the top of the returned result set.
exclude will exclude specified results from the returned result set.

Rule criteria

edit

The criteria for which this rule will match. Criteria is defined as type, metadata, and values. Allowed criteria types are:

Type	Match Requirements
`exact`	Rule metadata matches the specified value exactly.
`fuzzy`	Rule metadata matches the specified value within an allowed Levenshtein edit distance.
`prefix`	Rule metadata starts with the specified value.
`suffix`	Rule metadata ends with the specified value.
`contains`	Rule metadata contains the specified value.
`lt`	Rule metadata is less than the specified value.
`lte`	Rule metadata is less than or equal to the specified value.
`gt`	Rule metadata is greater than the specified value.
`gte`	Rule metadata is greater than or equal to the specified value.
`always`	Always matches for all rule queries.

Rule actions

edit

The actions to take when the rule matches a query:

ids will select the specified _ids.
docs will select the specified documents in the specified indices.

Use ids when searching over a single index, and docs when searching over multiple indices. ids and docs cannot be combined in the same query.

Add query rules

edit

You can add query rules using the Create or update query ruleset call. This adds a ruleset containing one or more query rules that will be applied to queries that match their specified criteria.

The following command will create a query ruleset called my-ruleset with two query rules:

The first rule will generate a Pinned Query pinning the _ids id1 and id2 when the query_string metadata value is a fuzzy match to either puggles or pugs and the user’s location is in the US.
The second rule will generate a query that excludes the _id id3 specifically from the my-index-000001 index and id4 from the my-index-000002 index when the query_string metadata value contains beagles.

resp = client.query_rules.put_ruleset(
    ruleset_id="my-ruleset",
    rules=[
        {
            "rule_id": "rule1",
            "type": "pinned",
            "criteria": [
                {
                    "type": "fuzzy",
                    "metadata": "query_string",
                    "values": [
                        "puggles",
                        "pugs"
                    ]
                },
                {
                    "type": "exact",
                    "metadata": "user_country",
                    "values": [
                        "us"
                    ]
                }
            ],
            "actions": {
                "ids": [
                    "id1",
                    "id2"
                ]
            }
        },
        {
            "rule_id": "rule2",
            "type": "exclude",
            "criteria": [
                {
                    "type": "contains",
                    "metadata": "query_string",
                    "values": [
                        "beagles"
                    ]
                }
            ],
            "actions": {
                "docs": [
                    {
                        "_index": "my-index-000001",
                        "_id": "id3"
                    },
                    {
                        "_index": "my-index-000002",
                        "_id": "id4"
                    }
                ]
            }
        }
    ],
)
print(resp)

const response = await client.transport.request({
  method: "PUT",
  path: "/_query_rules/my-ruleset",
  body: {
    rules: [
      {
        rule_id: "rule1",
        type: "pinned",
        criteria: [
          {
            type: "fuzzy",
            metadata: "query_string",
            values: ["puggles", "pugs"],
          },
          {
            type: "exact",
            metadata: "user_country",
            values: ["us"],
          },
        ],
        actions: {
          ids: ["id1", "id2"],
        },
      },
      {
        rule_id: "rule2",
        type: "exclude",
        criteria: [
          {
            type: "contains",
            metadata: "query_string",
            values: ["beagles"],
          },
        ],
        actions: {
          docs: [
            {
              _index: "my-index-000001",
              _id: "id3",
            },
            {
              _index: "my-index-000002",
              _id: "id4",
            },
          ],
        },
      },
    ],
  },
});
console.log(response);

PUT /_query_rules/my-ruleset
{
  "rules": [
    {
      "rule_id": "rule1",
      "type": "pinned",
      "criteria": [
        {
          "type": "fuzzy",
          "metadata": "query_string",
          "values": [ "puggles", "pugs" ]
        },
        {
          "type": "exact",
          "metadata": "user_country",
          "values": [ "us" ]
        }
      ],
      "actions": {
        "ids": [
          "id1",
          "id2"
        ]
      }
    },
    {
      "rule_id": "rule2",
      "type": "exclude",
      "criteria": [
        {
          "type": "contains",
          "metadata": "query_string",
          "values": [ "beagles" ]
        }
      ],
      "actions": {
        "docs": [
          {
            "_index": "my-index-000001",
            "_id": "id3"
          },
          {
            "_index": "my-index-000002",
            "_id": "id4"
          }
        ]
      }
    }
  ]
}

The API response returns a results of created or updated depending on whether this was a new or edited ruleset.

There is a limit of 100 rules per ruleset. This can be increased up to 1000 using the xpack.applications.rules.max_rules_per_ruleset cluster setting.

{
  "result": "created"
}

You can use the Get query ruleset call to retrieve the ruleset you just created, the List query rulesets call to retrieve a summary of all query rulesets, and the Delete query ruleset call to delete a query ruleset.

Search using query rules

edit

Once you have defined one or more query rulesets, you can search using these rulesets using the rule retriever or the rule query. Retrievers are the recommended way to use rule queries, as they will work out of the box with other reranking retrievers such as Reciprocal rank fusion.

Rulesets are evaluated in order, so rules in the first ruleset you specify will be applied before any subsequent rulesets.

An example query for the my-ruleset defined above is:

resp = client.search(
    index="my-index-000001",
    retriever={
        "rule": {
            "retriever": {
                "standard": {
                    "query": {
                        "query_string": {
                            "query": "puggles"
                        }
                    }
                }
            },
            "match_criteria": {
                "query_string": "puggles",
                "user_country": "us"
            },
            "ruleset_ids": [
                "my-ruleset"
            ]
        }
    },
)
print(resp)

const response = await client.search({
  index: "my-index-000001",
  retriever: {
    rule: {
      retriever: {
        standard: {
          query: {
            query_string: {
              query: "puggles",
            },
          },
        },
      },
      match_criteria: {
        query_string: "puggles",
        user_country: "us",
      },
      ruleset_ids: ["my-ruleset"],
    },
  },
});
console.log(response);

GET /my-index-000001/_search
{
  "retriever": {
    "rule": {
      "retriever": {
        "standard": {
          "query": {
            "query_string": {
              "query": "puggles"
            }
          }
        }
      },
      "match_criteria": {
        "query_string": "puggles",
        "user_country": "us"
      },
      "ruleset_ids": [ "my-ruleset" ]
    }
  }
}

This rule query will match against rule1 in the defined query ruleset, and will convert the organic query into a pinned query with id1 and id2 pinned as the top hits. Any other matches from the organic query will be returned below the pinned results.

It’s possible to have multiple rules in a ruleset match a single rule query. In this case, the rules are applied in the following order:

Where the matching rule appears in the ruleset
If multiple documents are specified in a single rule, in the order they are specified
If a document is matched by both a pinned rule and an exclude rule, the exclude rule will take precedence

You can specify reranking retrievers such as rrf or text_similarity_reranker in the rule query to apply query rules on already-reranked results. Here is an example:

resp = client.search(
    index="my-index-000001",
    retriever={
        "rule": {
            "match_criteria": {
                "query_string": "puggles",
                "user_country": "us"
            },
            "ruleset_ids": [
                "my-ruleset"
            ],
            "retriever": {
                "rrf": {
                    "retrievers": [
                        {
                            "standard": {
                                "query": {
                                    "query_string": {
                                        "query": "pugs"
                                    }
                                }
                            }
                        },
                        {
                            "standard": {
                                "query": {
                                    "query_string": {
                                        "query": "puggles"
                                    }
                                }
                            }
                        }
                    ]
                }
            }
        }
    },
)
print(resp)

const response = await client.search({
  index: "my-index-000001",
  retriever: {
    rule: {
      match_criteria: {
        query_string: "puggles",
        user_country: "us",
      },
      ruleset_ids: ["my-ruleset"],
      retriever: {
        rrf: {
          retrievers: [
            {
              standard: {
                query: {
                  query_string: {
                    query: "pugs",
                  },
                },
              },
            },
            {
              standard: {
                query: {
                  query_string: {
                    query: "puggles",
                  },
                },
              },
            },
          ],
        },
      },
    },
  },
});
console.log(response);

GET my-index-000001/_search
{
  "retriever": {
    "rule": {
      "match_criteria": {
        "query_string": "puggles",
        "user_country": "us"
      },
      "ruleset_ids": [
        "my-ruleset"
      ],
      "retriever": {
        "rrf": {
          "retrievers": [
            {
              "standard": {
                "query": {
                  "query_string": {
                    "query": "pugs"
                  }
                }
              }
            },
            {
              "standard": {
                "query": {
                  "query_string": {
                    "query": "puggles"
                  }
                }
              }
            }
          ]
        }
      }
    }
  }
}

This will apply pinned and excluded query rules on top of the content that was reranked by RRF.

« Search shard routing Search templates »