MLflow AI Gateway
One governed endpoint for many LLM providers, inside MLflow.
Part of the open-source MLflow platform: exposes several providers behind one endpoint, keeps provider keys in one place, splits traffic for A/B tests, chains failovers and logs requests for audit, with spend limits and budget alerts. Suits teams already using MLflow for evals and tracing.
- Vendor
- MLflow
- Category
- AI gateways
- Pricing
- Open source
- Open source
- Yes
- License
- Apache 2.0
- Platforms
- Python, Self-hosted
- Website
- mlflow.org/docs/latest/genai/governance/ai-gateway
Features
- Unified endpoint
- Central key storage
- Traffic splitting
- Budgets
Best for
- Enterprise
More ai gateways
- OpenRouter — OpenRouter. One API for hundreds of models.
- LiteLLM — BerriAI. Open-source, self-hosted gateway to 100+ LLM APIs in the OpenAI format.
- Portkey — Portkey. Production AI gateway with routing, guardrails, budgets and observability.
- Cloudflare AI Gateway — Cloudflare. Edge gateway for caching, rate-limiting and observing AI API calls.
- Vercel AI Gateway — Vercel. One endpoint for hundreds of models, with no markup on tokens.
- Azure API Management AI gateway — Microsoft. AI gateway policies built into Azure API Management.
- Kong AI Gateway — Kong. AI plugins that turn the Kong API gateway into a multi-LLM gateway.
- Apigee AI gateway — Google Cloud. AI gateway policies in Google Cloud’s Apigee API platform.