Skip to main content

OpenTelemetry v2

OpenTelemetry v2 (OTel v2) is LiteLLM Proxy's next-generation tracing. It gives you one clean trace per request that shows the whole story of a request — the incoming HTTP call, authentication, guardrails, the LLM call itself, and the internal database/cache work — all nested in a single tree.

It follows standard OpenTelemetry GenAI semantic conventions, so the traces it produces are readable in any OTel backend (Grafana Tempo, Jaeger, Honeycomb, Datadog, …) and come with ready-made presets for popular LLM observability tools (Arize, Phoenix, Langfuse, Weave, Langtrace, Levo, AgentOps).

Opt-in feature

OTel v2 is off by default. Nothing in it runs until you set LITELLM_OTEL_V2=true. It is separate from the existing OpenTelemetry integration — pick one. If you are moving from v1, see Migrating to OpenTelemetry v2.

What you get​

A single request to your proxy produces one trace that looks like this:

POST /v1/chat/completions                  ← HTTP request (server span)
├── auth /v1/chat/completions              ← authentication
│   ├── postgres get_key_object            ← DB lookups during auth
│   └── postgres get_team_membership
├── execute_guardrail presidio-pii         ← each guardrail that runs
├── chat gpt-4o                            ← the LLM call (model, tokens, cost)
└── batch_write_to_db                      ← spend/usage written to DB

Highlights:

  • One trace, end to end — the HTTP request, auth, guardrails, the LLM call, and DB writes all live in the same trace, correctly nested.
  • Rich GenAI attributes — every LLM-call span carries gen_ai.* attributes: model, provider, token usage, cost, finish reasons, request parameters, and more.
  • Standards-based — built on the official OpenTelemetry GenAI semantic conventions, so it works with any OTel-compatible backend.
  • Vendor presets — one line to ship traces to Arize, Phoenix, Langfuse, Weave, Langtrace, Levo, or AgentOps in the format each tool expects.
  • Safe by default — prompts and responses are not captured unless you explicitly opt in. Noisy routes (health checks, metrics scrapes, UI assets) are excluded automatically.
  • Distributed tracing — if your client sends a traceparent header, LiteLLM's spans nest inside your existing trace.

Getting started​

Set LITELLM_OTEL_V2=true in the proxy environment, then pick a destination below.

1. Send traces to any OTLP collector​

This path sends spans over OTLP (the OpenTelemetry Protocol) to a collector or backend you are already running at the endpoint below; if you do not have one yet, stay on the console exporter from the Quickstart until you do. Set the feature flag plus the standard OTEL_* environment variables in the proxy's environment. No config change is needed.

LITELLM_OTEL_V2=true
OTEL_EXPORTER="otlp_http"
OTEL_ENDPOINT="http://localhost:4318"

Pass auth headers your backend needs via OTEL_HEADERS:

OTEL_HEADERS="api-key=your-key,x-tenant=acme"

Then start the proxy as usual:

litellm --config config.yaml

Make a request, and you'll see one trace per request in your backend.

2. Send traces to a specific tool (presets)​

For LLM observability tools, use a preset. A preset knows the tool's endpoint and emits attributes in the schema that tool expects. To enable one, add its name to callbacks in your config and set the tool's credentials as env vars.

config.yaml
litellm_settings:
  callbacks: ["arize"]
LITELLM_OTEL_V2=true
ARIZE_SPACE_ID="your-space-id"
ARIZE_API_KEY="your-api-key"
ARIZE_PROJECT_NAME="your-project-name"   # recommended: names the project traces land in
Send to several backends at once

To send the same traces to multiple vendors, list each preset in callbacks and set each one's env vars. For example, Langfuse and Arize together:

config.yaml
litellm_settings:
  callbacks: ["langfuse_otel", "arize"]

Each preset adds its own destination, so your spans reach all of them in parallel, each in that tool's native format.

Preset reference​

Every preset turns into one exporter on a single shared tracer. The table lists, for each one, the callback name you put in callbacks, the credentials it reads, where it sends, the attribute vocabulary it adds on top of the canonical gen_ai.* keys, and whether it supports per-request (per-team/key) credentials.

PresetCallbackRequired env varsOptional env varsDestinationVocabularyPer-request creds
Arize AXarizeARIZE_SPACE_ID (ARIZE_SPACE_KEY deprecated), ARIZE_API_KEYARIZE_PROJECT_NAME (names the project traces land in), ARIZE_ENDPOINT (gRPC, default https://otlp.arize.com/v1), ARIZE_HTTP_ENDPOINT (HTTP)Arize AX platformOpenInferenceYes
Arize Phoenixarize_phoenixPHOENIX_API_KEY (Phoenix Cloud only; self-hosted needs none)PHOENIX_COLLECTOR_HTTP_ENDPOINT or PHOENIX_COLLECTOR_ENDPOINT (protocol inferred from the value), PHOENIX_PROJECT_NAMEPhoenix (self-hosted or Phoenix Cloud)OpenInferenceNo
Langfuselangfuse_otelLANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEYLANGFUSE_HOST (or LANGFUSE_OTEL_HOST; default https://us.cloud.langfuse.com, EU is https://cloud.langfuse.com), OTEL_IGNORE_CONTEXT_PROPAGATION (set true to drop inbound traceparent)Langfuse Cloud or self-hostedLangfuseYes
Weave (W&B)weave_otelWANDB_API_KEY, WANDB_PROJECT_ID (<entity>/<project>)WANDB_HOST (default https://trace.wandb.ai)Weights & Biases WeaveOpenInference + WeaveYes
Langtracelangtracenone of its own—Langtrace, via an OpenTelemetry Collector (Langtrace ingests JSON-only OTLP)LangtraceNo
LevolevoLEVOAI_API_KEY, LEVOAI_ORG_ID, LEVOAI_WORKSPACE_ID, LEVOAI_COLLECTOR_URL—Levo collectorcanonical gen_ai.* onlyNo
AgentOpsagentopsAGENTOPS_API_KEYAGENTOPS_SERVICE_NAME (default agentops), AGENTOPS_ENVIRONMENT (no default)AgentOps (https://otlp.agentops.ai/v1/traces)canonical gen_ai.* onlyNo

Notes:

  • Arize AX vs Arize Phoenix are different backends from the same company. AX (arize) is the hosted platform; Phoenix (arize_phoenix) is the open-source tracer you self-host or run on Phoenix Cloud. They use different credentials and endpoints, so pick the callback for the backend you actually run. You can also enable both at once to send to each.
  • Langtrace ingests JSON-only OTLP at a custom path, so litellm v2 (which sends protobuf to /v1/traces) cannot export to it directly. Route through an OpenTelemetry Collector that re-encodes to JSON; the langtrace preset only adds the Langtrace attribute schema to your spans. See the Langtrace tab above for the collector config.
  • Vocabulary is additive: every preset's spans always carry the canonical OpenTelemetry gen_ai.* attributes; the listed vocabulary is layered on top so the destination tool reads its native schema.

Seeing your traces​

Once a backend is configured with its preset, each request shows up in that tool's UI as a chat <model> span under the request root. Each tab below covers the vendor-specific gotchas (project mapping, endpoint variants, metadata keys) that trip people up.

What Arize renders​

Open your Arize project; the trace appears under the project named by ARIZE_PROJECT_NAME. The openinference mapper stamps the OpenInference vocabulary onto the LLM-call span alongside the canonical gen_ai.* keys, so Arize reads its native schema without dropping the canonical ones.

Attributes added by the openinference mapper​

AttributeRestates
openinference.span.kindFixed LLM
llm.model_name, llm.providermodel, provider
llm.token_count.prompt, completion, totalusage split
llm.invocation_parametersJSON blob of request params
llm.input_messages.{idx}.message.role, contentprompt (content capture on)
llm.output_messages.{idx}.message.role, contentresponse (content capture on)
input.value, output.valueJSON arrays of the same (content capture on)
llm.tools.{idx}.tool.name, description, json_schematool definitions

See the full OpenInference spec for the definitive vocabulary.

Setup notes​

  • ARIZE_SPACE_KEY is the deprecated name for ARIZE_SPACE_ID; the preset still reads it for backward compatibility, but prefer ARIZE_SPACE_ID in new configs.

LiteLLM trace in Arize

Capturing prompts & responses​

By default, OTel v2 records metadata only (model, tokens, cost, timing) and never writes prompt or response text to your traces. This is intentional — it keeps sensitive content out of your observability backend.

To capture message content, opt in explicitly:

# no_content (default) — never capture prompts/responses
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="no_content"

# span_only — write prompts/responses as attributes on spans
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="span_only"

# event_only — write prompts/responses on log events instead of span attributes
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="event_only"

# span_and_event — write content to both spans and events
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT="span_and_event"

The gate is enforced centrally, so it applies to every backend at once — a user request can never force its prompt into your backend while capture is disabled.

Span attributes​

Attributes come from a chain of mappers stamped onto each span in order. The canonical genai mapper is always applied first, the legacy compatibility mapper is on by default, and each preset adds one vendor mapper on top. Later mappers can override earlier ones; the same span therefore carries several vocabularies describing the same call.

The first two tables cover the LLM-call span in the canonical vocabulary. Sections below list the other span kinds, then what each vendor mapper adds.

LLM-call span, canonical gen_ai.* + litellm.*​

Request-side keys:

AttributeWhen set
gen_ai.operation.namealways (chat, text_completion, embeddings)
gen_ai.provider.namealways
gen_ai.request.modelalways (the user-facing model group name)
gen_ai.request.temperature, top_p, top_k, max_tokenswhen set on the request
gen_ai.request.frequency_penalty, presence_penalty, seedwhen set
gen_ai.request.stop_sequenceswhen set (string array)
gen_ai.tool.{idx}.name, description, parametersone set per tool definition
server.address, server.portwhen the provider endpoint is known

Response, usage, cost, identity:

AttributeWhen set
gen_ai.response.id, gen_ai.response.modelon success
gen_ai.response.finish_reasonson success (string array)
gen_ai.usage.input_tokens, gen_ai.usage.output_tokenson success
gen_ai.input.messages, gen_ai.output.messagescontent capture on
gen_ai.system_instructionscontent capture on, when a system prompt is present
litellm.call_idalways
litellm.provider.modelalways (the model string actually sent to the provider)
litellm.request.streamingwhen true
litellm.cost.totalon success
litellm.cost.input, output, cache_read, cache_creation, tool_usagewhen the source reported the breakdown
litellm.cost.original, discount_amount, discount_percent, margin_fixed_amount, margin_percent, margin_total_amountwhen reported

Status and errors:

  • On failure: the span records the standard exception event (exception.type, exception.message), sets error.type from the exception class, and sets its status to ERROR.
  • On success: the status is left UNSET (the semconv default, matching the FastAPI server span). Only a genuine error sets ERROR, so do not key an alert on a status of OK.

Other span kinds​

Guardrail span — uses the litellm.guardrail.* namespace: name, mode, status, provider, action, response, violation_categories, confidence_score, risk_score, masked_entity_count, duration, id, policy_template, detection_method. status is one of success, guardrail_intervened, guardrail_failed_to_respond, or not_run; a blocking guardrail_intervened or guardrail_failed_to_respond also sets span status to ERROR.

Datastore span (redis, postgres) — db.system.name, db.operation.name, litellm.service.name, litellm.service.call_type.

Internal service span — the litellm.service.* keys only (no db.*).

MCP tool-call span — gen_ai.operation.name=execute_tool, mcp.method.name, mcp.session.id, gen_ai.tool.name, litellm.mcp.server.name, litellm.call_id, litellm.cost.total. gen_ai.tool.call.arguments and gen_ai.tool.call.result are gated by the same content-capture setting as prompt content.

Root HTTP server span — the HTTP semconv keys http.request.method, http.route, http.response.status_code, url.path, stamped by the FastAPI instrumentation (not by any of LiteLLM's mappers).

Each vendor preset also composes one vendor-specific mapper on top of these canonical keys, so the destination reads the trace in its native schema. Those per-vendor tables live under the matching Seeing your traces tab.

Attribute conventions​

LiteLLM emits one canonical set of GenAI attributes and layers other vocabularies on top by adding a mapper; the active set is controlled by mapper_names, with genai always first. The legacy mapper is on by default (LITELLM_OTEL_LEGACY_COMPAT=true) and re-emits the same data under the older semconv-ai / Traceloop names, so dashboards built against those keep working through a migration. Turn it off with LITELLM_OTEL_LEGACY_COMPAT=false once your queries use the canonical keys. Vendor mappers (openinference, langfuse, weave, langtrace) are added by their presets and never replace the canonical keys.

The most common keys line up across vocabularies as follows:

Canonical (genai)Legacy (Traceloop)OpenInference
gen_ai.usage.input_tokensgen_ai.usage.prompt_tokensllm.token_count.prompt
gen_ai.usage.output_tokensgen_ai.usage.completion_tokensllm.token_count.completion
gen_ai.provider.namegen_ai.systemllm.provider
litellm.request.streamingllm.is_streamingn/a
gen_ai.request.modeln/allm.model_name

Request identity on every span​

LiteLLM writes a small allowlist of request-identity values into standard OpenTelemetry Baggage at the auth boundary. A custom span processor then copies those values onto every span in the trace, so a guardrail, datastore, or service span is filterable by team or key without LiteLLM re-stamping each one by hand.

By default the following keys are written onto every span:

KeyValue
litellm.team.idTeam UUID
litellm.team.aliasTeam display name
litellm.team.metadataTeam's free-form metadata, filtered to the sub-keys you allowlist
litellm.api_key.hashHash of the caller's virtual key
gen_ai.request.modelUser-facing model group name
litellm.provider.modelDispatched model on the provider

A separate set of request-metadata fields is written under the litellm.metadata.* namespace. Defaults:

litellm.metadata.user_api_key_org_id, litellm.metadata.user_api_key_user_id, litellm.metadata.user_api_key_alias, litellm.metadata.user_api_key_end_user_id, litellm.metadata.requester_ip_address.

Two defaults stay conservative for privacy. The end-user id is promotable but off by default at the top level (it identifies an individual); it appears under litellm.metadata.user_api_key_end_user_id, which callers who filter by user should enable. A team's free-form metadata is never emitted whole; only the sub-keys you allowlist leave the process, and the allowlist is empty by default.

Override any of these with the LITELLM_OTEL_BAGGAGE_PROMOTED_KEYS, LITELLM_OTEL_BAGGAGE_METADATA_KEYS, and LITELLM_OTEL_BAGGAGE_TEAM_METADATA_KEYS env vars (comma-separated), or the matching YAML lists under callback_settings.otel.

Metrics​

Alongside traces, OTel v2 can emit GenAI client metrics: histograms for call latency, token usage, and cost that your backend aggregates across requests. Like the rest of OTel v2 they stay off until you turn them on.

Set the flag in the proxy environment next to LITELLM_OTEL_V2:

LITELLM_OTEL_V2=true
LITELLM_OTEL_INTEGRATION_ENABLE_METRICS=true

Metrics ship through the exporter you already configured for traces. OTEL_EXPORTER (console, otlp_http, otlp_grpc), OTEL_ENDPOINT, and OTEL_HEADERS decide where the metric stream goes exactly as they do for spans, so the collector that receives your traces receives the metrics too.

What's recorded​

Each successful LLM call records the standard OpenTelemetry GenAI client metrics:

MetricUnitWhat it measures
gen_ai.client.operation.durationsWall-clock time for the whole LLM call
gen_ai.client.token.usage{token}Tokens consumed, split into input and output by the gen_ai.token.type attribute
gen_ai.usage.costUSDLiteLLM's computed cost for the call
gen_ai.server.time_to_first_tokensTime to the first streamed token (streaming calls)
gen_ai.server.time_per_output_tokensAverage time per output token
gen_ai.client.response.durationsProvider-side generation time
Renamed in this release

gen_ai.usage.cost, gen_ai.server.time_to_first_token, and gen_ai.server.time_per_output_token were previously emitted as gen_ai.client.token.cost, gen_ai.client.response.time_to_first_token, and gen_ai.client.response.time_per_output_token. The older spellings are not GenAI semantic conventions and no vendor dashboard queries them, so nothing prebuilt could chart LiteLLM's cost or latency. If you hand-built panels or alerts against the old names, repoint them at the names above

Every sample carries the same identity attributes as the matching span (operation, provider/system, request model, framework, and selected metadata.* fields), so you can group the histograms by model, provider, key, or team. These are the same six metrics the v1 OpenTelemetry integration emits, with identical names and units, so a dashboard built for one reads the other.

Control metric attribute cardinality​

By default every metric sample is stamped with the full identity attribute set, which includes per-request fields such as hidden_params and several metadata.* values. Those are close to unique per request, so each one multiplies the number of time series your backend tracks (one series per distinct attribute combination). At volume this explodes metric cardinality, and some backends, for example Splunk Observability Cloud, start throttling or dropping the metrics.

v2 reads the same filter v1 does, from callback_settings.otel.attributes in your config. Nest an attributes block there with either an include_list (allowlist; emit only the listed attributes) or an exclude_list (denylist; emit everything except the listed attributes). The two are mutually exclusive. The filter applies to metrics only; spans keep their full attribute set, so traces stay rich while metric cardinality stays bounded.

The block sits under callback_settings.otel. With LITELLM_OTEL_V2 set, listing otel in callbacks builds the v2 logger and reads this block (it builds the legacy v1 logger only when the flag is off); the block is also read on the default path when no otel callback is listed.

Unlike v1, v2 has no per-instance attributes field, so this global block is the only source. v2 also resolves the filter lazily on the first metric a request records rather than at boot, so a bad config (both lists set, or a forbidden name) surfaces on that first recorded request and editing the lists takes effect only after a restart. The filter is read only on the default OTLP path (callback name otel or unset); preset destinations such as arize, arize_phoenix, and langfuse_otel emit their metrics with the full attribute set, the same as in v1.

config.yaml
callback_settings:
  otel:
    attributes:
      exclude_list:
        - hidden_params
        - metadata.requester_metadata
        - metadata.requester_ip_address
        - metadata.spend_logs_metadata
        - metadata.mcp_tool_call_metadata
        - metadata.vector_store_request_metadata
        - metadata.prompt_management_metadata

When you want the smallest, most predictable attribute set, list exactly the attributes to keep with include_list. Anything not listed is dropped from metrics:

config.yaml
callback_settings:
  otel:
    attributes:
      include_list:
        - gen_ai.operation.name
        - gen_ai.system
        - gen_ai.request.model
        - gen_ai.framework
        - metadata.user_api_key_team_id
        - metadata.user_api_key_org_id

gen_ai.token.type is never filtered out. It is stamped on gen_ai.client.token.usage after the filter runs, so the input/output split survives whatever list you set, and naming it in either include_list or exclude_list is rejected.

Which routes are traced​

High-frequency, non-LLM routes are excluded by default so they don't flood your traces: health checks (/health*), the Prometheus scrape (/metrics), and static UI/docs assets (/ui, /docs, /redoc, /_next, /openapi.json, favicons, …).

To change the set, use the standard OpenTelemetry env var (comma-separated paths, substring-matched):

# Trace everything, including health checks
OTEL_PYTHON_FASTAPI_EXCLUDED_URLS=""

# Exclude only your own custom paths
OTEL_PYTHON_FASTAPI_EXCLUDED_URLS="/health,/internal"

Per-key / per-team destinations (multi-tenant)​

One proxy can serve many tenants and send each tenant's traces only to that tenant's own backend, so a team never sees another team's traces. The proxy admin owns the routing; a team or key just points at a destination by name and never handles another tenant's secrets.

Proxy admin                          Team admin
  creates a destination  ───────►      sees which ones reach their team
  (backend + secrets + scope)          (read-only, on the team's info page)
        │
        └──────────► at request time
              the proxy matches the caller against every
              destination's scope and sends the trace there

The idea in one minute​

There are two pieces.

A destination is a named place to send traces, created by the proxy admin. It reuses the same backends and credentials as the presets above: it holds which backend it is (langfuse_otel, arize, weave_otel, or a generic OTLP endpoint, meaning any backend that speaks the OpenTelemetry Protocol), the connection details and secrets for that backend, and an access scope that says which teams or organizations are allowed to use it. An organization here is a group of teams; a team belongs to one org.

The access scope is the whole of the routing decision. A team does not opt in and cannot turn a destination on or off; if the scope names the team, or the team's organization, or the destination is global, that team's traces go there. Teams see which destinations reach them as a read-only resolved_logging_exporters list on /team/info and /organization/info; the names are all they see, never the endpoints, the secrets, or the scope map itself.

At request time the proxy takes the team the calling key belongs to and that team's organization, keeps every destination whose scope includes one of them, builds each one from its stored values, and sends the request's trace to all of them. If nothing matches, the trace goes only to your normal global exporter from the sections above.

Who can change what​

Destinations are proxy-admin-owned end to end. Only a proxy admin can create one, edit its backend, host or secrets, or change its access scope; the /credentials endpoints answer 401 to anything else, so a team-scoped key cannot even list them. A team admin never handles another tenant's secrets because they never touch destinations at all.

ActionProxy adminOrg adminTeam admin
Create or delete a destinationYesNoNo
Edit a destination's backend, host, or secretsYesNoNo
Set the access scope: global, orgs, or teamsYesNoNo
See which destinations reach their own team or orgYesYes, names onlyYes, names only

Set it up in the UI​

This is the common path, and one thing has to be true before a team's traces flow: the destination's access scope must include that team, either directly, through its organization, or by being global. A proxy admin does all of it on one screen, Logging & Alerts, then Logging Callbacks.

Proxy admin, create the destination:

  1. Open the proxy UI and go to Logging & Alerts, then Logging Callbacks.
  2. Click Add Callback and pick the backend's scoped destination entry from the Callback dropdown, for example Langfuse OTEL (scoped destination). The plain Langfuse OTEL entry above it is the proxy-wide callback from the sections above, not a destination. Give it a Name, fill in the host and the secrets for that backend, and set the access scope: turn on Global for every team, or pick specific Teams or Organizations. The secret values are the same ones you would set as that preset's env vars, copied from the backend's own dashboard (for example, your Langfuse project's API keys); see the Preset reference for which fields each backend needs.
  3. Save. From now on the secrets and the Global/Org scope are admin-only; team admins can only attach the destination to teams already in its scope.

Adding a logging destination: the Callback dropdown set to Langfuse OTEL (scoped destination), with the name, host and secret fields above the Global, Teams and Organizations scope controls

The destinations you create appear in the Logging Callbacks list alongside your proxy-wide callbacks, each tagged with its access scope. Mode stays empty for a destination, since a destination has no success/failure mode of its own:

Active logging callbacks: one Global destination, one scoped to a team, one scoped to an organization, and one showing the Not active badge

A Not active badge means the destination cannot be built from the values it holds, usually because a required field is missing, so it receives no traces even where its scope would grant it. Fix the values and the badge clears.

To change who a destination reaches after creating it, open the row's actions menu and choose Edit scope. This edits only the access scope; the host and secrets stay where they are.

The Edit scope dialog for a destination, with the Global toggle off and the Teams multi-select holding one team

That is the whole setup. Traces from every team in the scope start flowing on the next request, with no step on the team's side.

To check a team is covered, open Teams, pick the team, and read its resolved_logging_exporters; it names every destination that will receive that team's traces. Other tenants' destinations never appear there.

Set it up over the API​

The UI calls these endpoints; you can use them directly. $ADMIN_KEY is a proxy-admin virtual key (mint one on the Virtual Keys page in the UI, or with /key/generate), <team-id> comes from the Teams page, and pk-... / sk-... are the backend's own keys from its dashboard. Creating the destination is the only write; there is no second call to turn it on.

Proxy admin creates a destination, here a Langfuse destination granted to one team:

curl -X POST http://localhost:4000/credentials \
  -H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
  -d '{
    "credential_name": "tenant-a-langfuse",
    "credential_values": {
      "langfuse_public_key": "pk-...",
      "langfuse_secret_key": "sk-...",
      "langfuse_host": "https://cloud.langfuse.com"
    },
    "credential_info": {
      "credential_type": "logging",
      "description": "langfuse_otel",
      "host": "https://cloud.langfuse.com",
      "access": { "teams": ["<team-id>"] }
    }
  }'

credential_type must be logging, and description names the backend. access accepts exactly global, teams and orgs; anything else is rejected with a 400, and per-key access is deliberately unsupported.

To widen or narrow the scope later, patch the same credential. credential_info is replaced wholesale rather than merged, so send the fields you want to keep:

curl -X PATCH http://localhost:4000/credentials/tenant-a-langfuse \
  -H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
  -d '{
    "credential_name": "tenant-a-langfuse",
    "credential_info": {
      "credential_type": "logging",
      "description": "langfuse_otel",
      "access": { "teams": ["<team-id>", "<other-team-id>"] }
    },
    "credential_values": {}
  }'

Confirm it landed by reading the team back; resolved_logging_exporters names every destination that will receive that team's traces:

curl "http://localhost:4000/team/info?team_id=<team-id>" \
  -H "Authorization: Bearer $ADMIN_KEY"

Backends and the fields each one needs​

The admin fills these into the destination's secret fields; the values come from the backend's own dashboard, the same as the preset env vars in the Preset reference. Anything OTLP-compatible that is not one of the first three uses generic.

Backend (description)Secret fields
langfuse_otellangfuse_public_key, langfuse_secret_key, langfuse_host (optional; defaults to Langfuse US cloud)
arizearize_space_id (or arize_space_key), arize_api_key, arize_project_name; arize_endpoint optional
weave_otelwandb_api_key, weave_project_id (optional); weave_endpoint optional
genericotel_endpoint (required), otel_headers (optional, key=value,key2=value2)

Good to know​

Resolution is default-deny. An access that grants nobody, whether it is absent, {}, or names only teams that do not exist, sends nothing rather than leaking a trace to the wrong tenant. The same holds for a destination whose values do not build; it is reported as Not active in the UI, is left out of resolved_logging_exporters, and receives nothing.

Set access.global to true to reach every team on the proxy without naming them. It is the one scope that does not need an org or team id, and it is still admin-only, so no team can grant itself global reach.

Destinations are additive with the global exporter, not a replacement for it. A request whose team matches two destinations sends its trace to both of them and to your configured exporter; two destinations that resolve to the same endpoint and headers collapse to one send.

This routing applies to traces only. The GenAI client metrics (see Metrics) still go to your single globally-configured exporter, not to per-tenant destinations.

Distributed tracing​

If the incoming request has a W3C traceparent header, LiteLLM continues that trace instead of starting a new one. Your LiteLLM spans then appear inline inside whatever distributed trace your application already has — so you can follow a request from your app, through the proxy, to the LLM provider, in one view.

Configuration reference​

All values are environment variables. Boolean flags accept true/false.

VariableDefaultPurpose
LITELLM_OTEL_V2falseMaster switch. OTel v2 does nothing until this is true.
OTEL_EXPORTER (alias OTEL_EXPORTER_OTLP_PROTOCOL)consoleExporter kind: console, otlp_http, otlp_grpc.
OTEL_ENDPOINT (alias OTEL_EXPORTER_OTLP_ENDPOINT)noneOTLP collector URL. Setting an endpoint implies otlp_http unless you override OTEL_EXPORTER.
OTEL_HEADERS (alias OTEL_EXPORTER_OTLP_HEADERS)noneComma-separated key=value auth headers for your backend.
OTEL_SERVICE_NAMElitellmservice.name resource attribute shown in your backend.
OTEL_ENVIRONMENT_NAMEnonedeployment.environment resource attribute (e.g. production).
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENTno_contentPrompt/response capture: no_content, span_only, event_only, span_and_event.
OTEL_PYTHON_FASTAPI_EXCLUDED_URLShealth/metrics/UI routesComma-separated paths to exclude from tracing (substring match). Set to "" to trace everything.
LITELLM_OTEL_INTEGRATION_ENABLE_METRICSfalseAlso emit the GenAI client metrics (duration, token usage, cost, streaming timings). See Metrics.
LITELLM_OTEL_LEGACY_COMPATtrueAlso emit attributes under the older Traceloop key names. See Attribute conventions.

The full set of keys on each span kind is in Span attributes.

Troubleshooting​

No traces showing up?

  1. Confirm LITELLM_OTEL_V2=true is set in the proxy's environment.
  2. Try OTEL_EXPORTER="console" first — if spans print to stdout, the problem is your exporter endpoint/headers, not LiteLLM.
  3. Make sure you hit an LLM route (e.g. /v1/chat/completions). Health checks and UI routes are excluded by default.
  4. Check that opentelemetry-instrumentation-fastapi is installed (see Requirements).

Only see the LLM call but no auth/postgres/server span? Those server and DB spans require the FastAPI instrumentation package — install opentelemetry-instrumentation-fastapi.

I see metadata but no prompts/responses. That's the default. Set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=span_only to capture content.

Support​

For questions, open an issue at BerriAI/litellm.

🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →