Skip to main content

OpenTelemetry

This document describes how OpenTelemetry is wired in the API (apps/api) and what telemetry is emitted today.

Where setup happens​

OpenTelemetry bootstrap lives in apps/api/src/shared/kernel/otel.ts via setupOtel(app).

apps/api/src/index.ts calls it early:

  • creates Fastify app
  • calls setupOtel(app)
  • keeps returned tracerProvider, meterProvider, and httpRequestDurationMs
  • shuts both providers down during graceful shutdown

Tracing​

Tracing is exported with OTLP HTTP/protobuf via OTLPTraceExporter.

  • Exporter URL: AXIOM_OTLP_TRACES_URL (default: https://api.axiom.co/v1/traces)
  • Headers:
    • Authorization: Bearer <AXIOM_EVENTS_TOKEN>
    • X-Axiom-Dataset: <AXIOM_EVENTS_DATASET>
  • Provider: NodeTracerProvider
  • Span processor: BatchSpanProcessor

Fastify request spans are created by @fastify/otel instrumentation plugin, registered in setupOtel.

Current instrumentation options:

  • ignorePaths: "/health" (health endpoint is excluded from instrumentation)

Metrics​

Metrics are exported with OTLP HTTP/protobuf via OTLPMetricExporter.

  • Exporter URL: AXIOM_OTLP_METRICS_URL (default: https://api.axiom.co/v1/metrics)
  • Headers:
    • Authorization: Bearer <AXIOM_METRICS_TOKEN>
    • x-axiom-metrics-dataset: <AXIOM_METRICS_DATASET>
    • contentType: application/x-protobuf

Metric export is driven by PeriodicExportingMetricReader:

  • export interval: 10_000ms
  • export timeout: 8_000ms

Custom metric emitted by API​

The app creates one histogram:

  • Name: http.server.request.duration
  • Unit: ms
  • Description: duration of incoming HTTP requests

Recorded in index.ts onResponse hook with attributes:

  • http.request.method
  • http.route
  • http.response.status_code

OPTIONS requests are skipped in this hook, so those are not included in this custom histogram.

Resource attributes​

Both tracer and meter providers share the same OpenTelemetry resource:

  • service.name = OTEL_SERVICE_NAME (fallback: kinky-api)

Request correlation and span enrichment​

In index.ts:

  • onRequest resolves or creates requestId (from x-request-id header or crypto.randomUUID()).
  • The same ID is attached to the active span as request.id.
  • onResponse and setErrorHandler read current span context and include trace_id/span_id in logs.

This gives practical trace-log correlation even though logs are sent by Pino separately.

Diagnostics (SDK internal logging)​

If OTEL_LOG_LEVEL is set, setupOtel enables OpenTelemetry diagnostic logging using DiagConsoleLogger.

Supported values:

  • all
  • verbose
  • debug
  • info
  • warn
  • error
  • none

If unset, OpenTelemetry diag logging is not explicitly enabled.

Required environment variables​

Validated at startup in apps/api/src/shared/kernel/env.ts:

  • AXIOM_OTLP_TRACES_URL
  • AXIOM_EVENTS_TOKEN
  • AXIOM_EVENTS_DATASET
  • OTEL_SERVICE_NAME
  • AXIOM_OTLP_METRICS_URL
  • AXIOM_METRICS_TOKEN
  • AXIOM_METRICS_DATASET

Operational notes​

  • If OTEL credentials/endpoints are invalid, exports will fail at runtime (the app still starts because providers are created regardless).
  • Graceful shutdown calls tracerProvider.shutdown() and meterProvider.shutdown() to flush telemetry before process exit.