Datadog LLM Observability
LLM and agent tracing inside the Datadog APM stack.
Traces LLM calls and agent steps end to end, tracks cost, latency and errors, and runs quality and security evaluations, all alongside existing Datadog infrastructure and application monitoring.
- Vendor
- Datadog
- Category
- Observability & evals
- Pricing
- Paid
- Open source
- No
- Platforms
- Cloud, Python, TypeScript
- Launched
- 2024
- Website
- datadoghq.com/product/llm-observability
Features
- Agent tracing
- Cost & latency tracking
- Quality & security evals
Best for
- Enterprise
More observability & evals
- LangSmith — LangChain. Tracing, evals and monitoring for LLM apps.
- Langfuse — Langfuse. Open-source LLM engineering platform.
- MLflow — Databricks (Linux Foundation project). Open-source tracing, evaluation and prompt management for GenAI and ML.
- Braintrust — Braintrust. The evals platform for AI products.
- promptfoo — promptfoo. Test and red-team your LLM apps.
- Arize Phoenix — Arize AI. Open-source AI observability and evaluation.
- W&B Weave — Weights & Biases. Track and evaluate LLM applications.
- Helicone — Helicone. LLM observability via a one-line proxy.