Set up Observability
This page is about agent observability — exporting agent traces to your own observability platform so you can debug agent behavior and monitor production workloads. It's configured entirely from the application; there's no separate infrastructure to deploy.
This is the first-time (self-hosted) setup. For the day-to-day reference once you're running, see Observability in Administration.
This covers agent traces only. Infrastructure observability — metrics, logs, and traces for the underlying compute, network, and data services — is your responsibility as part of running the deployment in your own cloud, using whatever tooling your platform team already uses.
Configure agent observability under Configuration > Observability in the application.
Supported vendors
Pick a Vendor in the form and fill in its fields. Each vendor exports over OTLP (OpenTelemetry):
| Vendor | Fields | Vendor docs |
|---|---|---|
| LangSmith | URL (OTLP endpoint, e.g. https://api.smith.langchain.com/otel), Project Name, API Key | Trace with OpenTelemetry (opens in a new tab) |
| Grafana Cloud | URL (OTLP endpoint), Grafana Instance ID, API Token, optional Stack Name, optional additional headers | Send data to the Grafana Cloud OTLP endpoint (opens in a new tab) |
| OTLP (Username/Password) | URL, Username, Password — for any OTLP endpoint using basic auth | OpenTelemetry OTLP (opens in a new tab) |
| OTLP (Custom Headers) | URL, custom HTTP headers — for any OTLP endpoint using header-based auth | OpenTelemetry OTLP (opens in a new tab) |
Quickest to set up: LangSmith — just an OTLP endpoint, a project name, and an API key. The two OTLP options are generic, so you can point them at any OpenTelemetry-compatible backend (your own Collector, Jaeger, Honeycomb, etc.).
Configure it
Step 1: Pick a vendor and fill in the fields
Under Configuration > Observability, choose your Vendor and enter its fields. Get the endpoint URL and credentials from the vendor's documentation (linked above).
Step 2: Test
Click Test — it sends a heartbeat to validate the configuration. Fix any reported errors before saving.
Step 3: Update
Click Update to save. New agent traces will start exporting to your platform.
For the two OTLP vendors you can also set a Trace UI Type (Grafana or Jaeger) and a Trace View URL to deep-link from a conversation straight to its trace in your backend. Grafana Cloud uses the optional Stack Name for the same purpose.
Support mode
The form has a Support mode toggle. When on, the integration also forwards HTTP, database, and infrastructure traces to your vendor — not just agent traces. This gives the fullest picture for deep debugging, which is useful when troubleshooting with Sema4.ai support.
Enable Support mode only while actively debugging. It produces much more data than agent traces alone. Turn it off again when you're done.
Network egress
Observability data leaves your deployment over the network, so allow outbound HTTPS (443) to your vendor's endpoint:
- LangSmith —
api.smith.langchain.com - Grafana Cloud — your stack's OTLP gateway host (e.g.
otlp-gateway-<region>.grafana.net) - OTLP — whatever endpoint host you configured (your Collector or vendor ingest URL)
If the Test heartbeat fails with valid credentials, suspect blocked egress first. See Add networking rules.