OpenTelemetry bridge

edit

Integration with the OpenTelemetry Tracing API was added as experimental in v3.34.0. Integration with the OpenTelemetry Metrics API was added as experimental in v3.45.0.

The Elastic APM OpenTelemetry bridge allows one to use the vendor-neutral OpenTelemetry API (@opentelemetry/api) in your code, and have the Elastic Node.js APM agent handle those API calls. This allows one to use the Elastic APM agent for tracing and metrics without any vendor lock-in to the APM agent’s own public API when adding manual tracing or custom metrics.

Using the OpenTelemetry Tracing API

edit

① First, you will need to add the Elastic APM agent and OpenTelemetry API dependencies to your project. The minimum required OpenTelemetry API version is 1.0.0; see the OpenTelemetry compatibility section for the current maximum supported API version. For example:

npm install --save elastic-apm-node @opentelemetry/api

② Second, you will need to configure and start the APM agent. This can be done completely with environment variables (so that there is no need to touch your application code):

export ELASTIC_APM_SERVER_URL='<url of your APM server>'
export ELASTIC_APM_SECRET_TOKEN='<secret token for your APM server>'  # or ELASTIC_APM_API_KEY=...
export ELASTIC_APM_OPENTELEMETRY_BRIDGE_ENABLED=true 
export NODE_OPTIONS='-r elastic-apm-node/start.js'  # Tell node to preload and start the APM agent
node my-app.js

Future versions may drop this config var and enable usage of the tracing API by default.

Or, alternatively, you can configure and start the APM agent at the top of your application code:

require('elastic-apm-node').start({
    serverUrl: '<url of your APM server>',
    secretToken: '<secret token for your APM server>', // or, apiKey: '<your API key>'
    opentelemetryBridgeEnabled: true
});

// Application code ...

See the full APM agent configuration reference for other configuration options.

③ Finally, you can use the OpenTelemetry API for any manual tracing in your code. For example, the following script uses Tracer#startActiveSpan() to trace an outgoing HTTPS request:

const https = require('https')
const otel = require('@opentelemetry/api')
const tracer = otel.trace.getTracer('trace-https-request')

tracer.startActiveSpan('makeRequest', span => {
  https.get('https://httpstat.us/200', (response) => {
    console.log('STATUS:', response.statusCode)
    const body = []
    response.on('data', (chunk) => body.push(chunk))
    response.on('end', () => {
      console.log('BODY:', body.toString())
      span.end()
    })
  })
})

The APM agent source code repository includes some examples using the OpenTelemetry tracing bridge.

Using the OpenTelemetry Metrics API

edit

① As above, install the needed dependencies. The minimum required OpenTelemetry API version is 1.3.0 (the version when metrics were added); see the OpenTelemetry compatibility section for the current maximum supported API version. For example:

npm install --save elastic-apm-node @opentelemetry/api

② Configure and start the APM agent. This can be done completely with environment variables — as shown below — or in code. (See Starting the agent and the full APM agent configuration reference for other configuration options.)

export ELASTIC_APM_SERVER_URL='<url of your APM server>'
export ELASTIC_APM_SECRET_TOKEN='<secret token for your APM server>'  # or ELASTIC_APM_API_KEY=...
export NODE_OPTIONS='-r elastic-apm-node/start.js'  # Tell node to preload and start the APM agent
node my-app.js

③ Finally, you can use the OpenTelemetry Metrics API, to create metrics and the APM agent will periodically ship those metrics to your Elastic APM deployment where you can visualize them in Kibana.

// otel-metrics-hello-world.js 
const { createServer } = require('http')
const otel = require('@opentelemetry/api')

const meter = otel.metrics.getMeter('my-meter')
const numReqs = meter.createCounter('num_requests', { description: 'number of HTTP requests' })

const server = createServer((req, res) => {
  numReqs.add(1)
  req.resume()
  req.on('end', () => {
    res.end('pong\n')
  })
})
server.listen(3000, () => {
  console.log('listening at http://127.0.0.1:3000/')
})

The full example is here.

Using the OpenTelemetry Metrics SDK

edit

The Elastic APM agent also supports exporting metrics to APM server when the OpenTelemetry Metrics SDK is being used directly. You might want to use the OpenTelemetry Metrics SDK to use a View to configure histogram bucket sizes, to setup a Prometheus exporter, or for other reasons. For example:

// use-otel-metrics-sdk.js 
const otel = require('@opentelemetry/api')
const { MeterProvider } = require('@opentelemetry/sdk-metrics')
const { PrometheusExporter } = require('@opentelemetry/exporter-prometheus')

const exporter = new PrometheusExporter({ host: '127.0.0.1', port: 3001 })
const meterProvider = new MeterProvider()
meterProvider.addMetricReader(exporter)
otel.metrics.setGlobalMeterProvider(meterProvider)

const meter = otel.metrics.getMeter('my-meter')
const latency = meter.createHistogram('latency', { description: 'Response latency (s)' })
// ...

The full example is here.

OpenTelemetry Metrics configuration

edit

A few configuration options can be used to control OpenTelemetry Metrics support.

Bridge architecture

edit

The OpenTelemetry Tracing bridge works similarly to the OpenTelemetry Node.js Trace SDK. It registers Tracer and ContextManager providers with the OpenTelemetry API. Subsequent @opentelemetry/api calls in user code will use those providers. The APM agent translates from OpenTelemetry to Elastic APM semantics and sends tracing data to your APM server for full support in Elastic Observability’s APM app.

Some examples of semantic translations: The first entry span of a service (e.g. an incoming HTTP request) will be converted to an Elasic APM Transaction, subsequent spans are mapped to Elastic APM `Span`s. OpenTelemetry Span attributes are translated into the appropriate fields in Elastic APM’s data model.

The only difference, from the user’s point of view, is in the setup of tracing. Instead of setting up the OpenTelemetry JS SDK, one sets up the APM agent as described above.


The OpenTelemetry Metrics support, is slightly different. If your code uses just the Metrics API, then the APM agent provides a full MeterProvider so that metrics are accumulated and sent to APM server. If your code uses the Metrics SDK, then the APM agents adds a MetricReader to your MeterProvider to send metrics on to APM server. This allows you to use the APM agent as either an easy setup for using metrics or in conjunction with your existing OpenTelemetry Metrics configuration.

Caveats

edit

Not all features of the OpenTelemetry API are supported. This section describes any limitations and differences.

Tracing
edit
  • Span Link Attributes. Adding links when starting a span is supported, but any added span link attributes are silently dropped.
  • Span events (Span#addEvent()) are not currently supported. Events will be silently dropped.
  • Propagating baggage within or outside the process is not supported. Baggage items are silently dropped.
Metrics
edit
  • Metrics exemplars are not supported.
  • Summary metrics are not supported.
  • Exponential Histograms are not yet supported.
  • The sum, count, min and max within the OpenTelemetry histogram data are discarded.
  • The default histogram bucket boundaries are different from the OpenTelemetry default. They provide better resolution. They can be configured with the customMetricsHistogramBoundaries configuration option.
  • Metrics label names are dedotted (s/\./_/g) in APM server to avoid possible mapping collisions in Elasticsearch.
  • The default Aggregation Temporality used differs from the OpenTelemetry default — preferring delta-temporality (nicer for visualizing in Kibana) to cumulative-temporality.

Metrics support requires an APM server >=7.11 — for earlier APM server versions, metrics with label names including ., *, or " will get dropped.

Logs
edit

The OpenTelemetry Logs API is currently not support — only the Tracing and Metrics APIs.