AI Gateway
A proxy layer that gives an application a single API to route requests across many LLM providers, with fallbacks, load balancing, caching, and cost/usage tracking.
Gateways like Portkey and LiteLLM decouple an application from any one model vendor: if a provider has an outage or a team wants to A/B test models, the gateway handles routing and failover without application code changes. Most also enforce guardrails and per-team spend limits, making them as much a governance tool as a routing one.
Last verified: