Adding custom URLs to machine learning results

edit

When you create an advanced job or edit any job in Kibana, you can optionally attach one or more custom URLs.

The custom URLs provide links from the anomalies table in the Anomaly Explorer or Single Metric Viewer window in Kibana to Kibana dashboards, the Discovery page, or external websites. For example, you can define a custom URL that provides a way for users to drill down to the source data from the results set.

When you edit a job in Kibana, it simplifies the creation of the custom URLs for Kibana dashboards and the Discover page and it enables you to test your URLs. For example:

Edit a job to add a custom URL

For each custom URL, you must supply the URL and a label, which is the link text that appears in the anomalies table. You can also optionally supply a time range. For example, these are the values that are added for My link 1:

An example of a label and URL

As in this case, the custom URL can contain dollar sign delimited tokens, which are populated when you click the link in the anomalies table. In this example, the custom URL contains $earliest$, $latest$, and $service$ tokens, which pass the beginning and end of the time span of the selected anomaly and the pertinent service field value to the target page. If you were interested in the following anomaly, for example:

An example of the custom URL links in the Anomaly Explorer anomalies table

…​clicking My Link 1 opens the Discover page and shows results for the service and date that were identified in the anomaly:

An example of the results on the Discover page

Since we specified a time range of 2 hours, the time filter restricts the results to the time period two hours before and after the anomaly.

You can also specify these custom URL settings when you create or update jobs by using the machine learning APIs.

String substitution in custom URLs

edit

You can use dollar sign ($) delimited tokens in a custom URL. These tokens are substituted for the values of the corresponding fields in the anomaly records. For example, for a configured URL of http://my.datastore.com/dashboards?user=$user_name$, the value of the user_name field in the anomaly record is substituted into the $user_name$ token when you click the link in the anomalies table.

Not all fields in your source data exist in the anomaly results. If a field is specified in the detector as the field_name, by_field_name, over_field_name, or partition_field_name, for example, it can be used in a custom URL. A field that is only used in the categorization_field_name property, however, does not exist in the anomaly results.

The following keywords can also be used as tokens for string substitution in a custom URL: $earliest$; $latest$; $mlcategoryregex$; $mlcategoryterms$.

The $earliest$ and $latest$ tokens pass the beginning and end of the time span of the selected anomaly to the target page. The tokens are substituted with date-time strings in ISO-8601 format. If you selected an interval of 1 hour for the anomalies table, these tokens use one hour on either side of the anomaly time as the earliest and latest times. The same is also true if the interval is set to Auto and a one hour interval was chosen. You can override this behavior by using the time_range setting.

The $mlcategoryregex$ and $mlcategoryterms$ tokens pertain to jobs where you are categorizing field values. For more information about this type of analysis, see Categorizing log messages.

The $mlcategoryregex$ token passes the regular expression value of the category of the selected anomaly, as identified by the value of the mlcategory field of the anomaly record.

The $mlcategoryterms$ token likewise passes the terms value of the category of the selected anomaly. Each categorization term is prefixed by a plus (+) character, so that when the token is passed to a Kibana dashboard, the resulting dashboard query seeks a match for all of the terms of the category.

For example, the following API updates a job to add a custom URL that uses $earliest$, $latest$, and $mlcategoryterms$ tokens:

POST _xpack/ml/anomaly_detectors/sample_job/_update
{
  "custom_settings": {
        "custom_urls": [
          {
            "url_name": "test-link1",
            "time_range": "1h",
            "url_value": "http://localhost:5601/app/kibana#/discover?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:'$earliest$',mode:quick,to:'$latest$'))&_a=(columns:!(_source),index:AV3OWB68ue3Ht69t29aw,interval:auto,query:(query_string:(analyze_wildcard:!t,query:'$mlcategoryterms$')),sort:!(time,desc))"
          }
        ]
      }
}

When you click this custom URL in the anomalies table in Kibana, it opens up the Discover page and displays source data for the period one hour before and after the anomaly occurred. Since this job was categorizing log messages, some $mlcategoryterms$ token values that were passed to the target page for an example anomaly are as follows:

A query for category terms on the Discover page in Kibana
  • The custom URL links in the anomaly tables use pop-ups. You must configure your web browser so that it does not block pop-up windows or create an exception for your Kibana URL.
  • When creating a link to a Kibana dashboard, the URLs for dashboards can be very long. Be careful of typos, end of line characters, and URL encoding. Also ensure you use the appropriate index ID for the target Kibana index pattern.
  • If you use an influencer name for string substitution, keep in mind that it might not always be available in the analysis results and the URL is invalid in those cases. There is not always a statistically significant influencer for each anomaly.
  • The dates substituted for $earliest$ and $latest$ tokens are in ISO-8601 format and the target system must understand this format.
  • If the job performs an analysis against nested JSON fields, the tokens for string substitution can refer to these fields using dot notation. For example, $cpu.total$.
  • Elasticsearch source data mappings might make it difficult for the query string to work. Test the custom URL before saving the job configuration to check that it works as expected, particularly when using string substitution.