Switchyard vs LiteLLM vs Portkey: 2026 Best AI Gateway Comparison

A coding agent is failing when one provider throttles, while a second model does not support the same tool or streaming behavior.

Fastest decision: evaluate Switchyard first for local coding-agent proxies and stage routing, LiteLLM first for broad provider compatibility and mature proxy functions, and Portkey first when managed observability and governance are the priority.

Last updated August 14, 2026. This comparison was checked against the official Switchyard repository, LiteLLM documentation, LiteLLM repository, Portkey Gateway documentation, Portkey documentation, and related release records. Gateway capabilities can change, so production teams should recheck the relevant documentation before switching traffic.

This guide is for teams that need one API surface across multiple models, platform engineers managing credentials and usage records, and developers routing Claude Code or other coding agents through shared infrastructure.

Start with the decision matrix

The three products solve related problems, but they do not have the same center of gravity.

Choose Switchyard when the request path is agent-first

The official NVIDIA-NeMo Switchyard repository describes a Python proxy that translates between OpenAI Chat, Anthropic Messages, and OpenAI Responses formats. It is designed to route coding-agent traffic to vLLM, NVIDIA NIM, Ollama, or other OpenAI-compatible endpoints. The repository also describes profile-backed routing, A/B benchmarking, and classifier-based stage-router escalation. Read the official Switchyard repository.

That makes Switchyard the most direct candidate when the main problem is:

• connecting Claude Code to a local or private model;
• preserving an agent’s native protocol while changing the backend;
• sending simple work to a weaker model and escalating harder work;
• testing several models without rewriting the client;
• keeping routing logic close to the development environment.

The limitation is equally important: a focused routing proxy is not automatically a complete platform governance product. Credential lifecycle, team budgets, audit workflows, retention policies, and dashboard operations still need to be verified for the exact deployment.

Choose LiteLLM when compatibility is the first constraint

LiteLLM’s official documentation presents both a Python SDK and a proxy server. The documentation states that LiteLLM supports more than 100 LLMs through a common OpenAI-style input and output format, with provider translation across endpoints such as chat completions, responses, embeddings, images, audio, and batches. It also documents retries, fallbacks, spend tracking, project budgets, virtual keys, authentication, rate limiting, and observability callbacks. Review LiteLLM’s official documentation.

LiteLLM deserves the first evaluation when:

• the application may switch among many hosted and private providers;
• the platform team needs a central proxy rather than a library inside one application;
• multiple projects require separate keys, budgets, and usage attribution;
• the team already uses callback-based monitoring;
• the gateway must expose a familiar API while translating provider-specific requests.

Its broad surface area creates a maintenance trade-off. A team must distinguish the open-source proxy, the Python SDK, and any commercial or enterprise functions instead of treating “LiteLLM” as one uniform feature tier.

Choose Portkey when operations matter more than minimal ownership

Portkey’s official AI Gateway documentation lists universal API access, fallbacks, conditional routing, retries, circuit breakers, load balancing, canary testing, budget limits, rate limits, caching, and MCP support. Its documentation also separates the open-source gateway from hosted and enterprise deployment paths. See Portkey’s official AI Gateway documentation.

Portkey is the stronger candidate when the team needs:

• a managed control plane for configurations and analytics;
• shared observability across gateway requests and application traces;
• governance workflows for teams, users, budgets, and limits;
• routing policies that need to be operated by more than one engineering group;
• a path from hosted gateway usage to self-hosted or hybrid deployment.

Portkey’s enterprise architecture documentation describes a stateless gateway that can scale horizontally while a hosted control plane manages administration, configurations, integrations, providers, and API keys. That separation can simplify operations, but it also means that data flow, log storage, control-plane dependency, and residency requirements must be approved before production use. Check Portkey’s hybrid deployment architecture.

Decision rule: if the primary acceptance test is “Can Claude Code reach a private model with controlled stage routing?”, begin with Switchyard. If it is “Can one proxy cover many providers and teams?”, begin with LiteLLM. If it is “Can platform operators manage routing, logs, budgets, and governance from a shared workflow?”, begin with Portkey.

Check protocol behavior before counting integrations

A gateway that accepts a basic chat request may still fail inside a coding agent. The meaningful test is not whether the first request returns text. The test is whether the client keeps working when tools, streaming, structured output, provider metadata, and errors are involved.

Switchyard’s official repository explicitly names translation among Anthropic Messages, OpenAI Chat, and OpenAI Responses formats. That is relevant to Claude Code because the client can preserve its expected API shape while the proxy sends traffic to another backend. The team should still validate tool-call identifiers, streamed event ordering, stop reasons, usage fields, and provider-specific headers in the exact agent version being used.

LiteLLM documents several API families and response behaviors, including streaming and exception handling. Its broader provider surface is useful for applications that mix chat, responses, embeddings, image generation, audio, and batch workloads. The integration advantage is strongest when the application can stay close to a common API. It becomes less certain when an upstream provider exposes a feature that has no clean equivalent in the common format.

Portkey also exposes a universal API and documents multimodal access, tool-oriented workflows, retries, and MCP support. The team should test whether the hosted and self-hosted paths expose the same behavior, because open-source gateway capability and hosted platform capability are not necessarily identical.

Use this acceptance checklist before selecting a winner:

• [ ] A normal text request succeeds through every planned backend.
• [ ] Streaming sends complete events in the order expected by the client.
• [ ] Tool calls preserve names, arguments, identifiers, and final results.
• [ ] Structured output is rejected or retried safely when a model cannot support it.
• [ ] Provider extensions do not disappear silently during translation.
• [ ] Authentication errors are not treated as retryable capacity errors.
• [ ] A mid-stream failure produces a known application outcome.
• [ ] Usage, latency, selected backend, and error type reach the logging system.

Compare model and backend coverage by maintenance ownership

Model lists change quickly. A documentation page that names a provider proves current documented support, but it does not prove that every adapter has equal test coverage or long-term maintenance.

LiteLLM has the widest documented compatibility claim among the three in the sources reviewed, including hosted providers, Azure OpenAI, Vertex AI, NVIDIA, Hugging Face, Ollama, OpenRouter, and other endpoints. The official documentation also shows the proxy connecting model aliases to provider-specific deployment parameters. That makes it attractive for a central platform that expects backend churn.

Switchyard’s documented emphasis is different. It focuses on translating agent-facing protocols and routing traffic toward local, private, or OpenAI-compatible inference endpoints. This is valuable for teams that already know which inference servers they operate. It is less useful to assume that a focused agent router should replace a full provider catalog without testing each required backend.

Portkey’s Gateway repository describes routing to more than 1,600 language, vision, audio, and image models. That figure is a documented coverage claim, not a guarantee that every model supports the same tools, context behavior, structured output, or enterprise controls. Review the official Portkey Gateway repository.

For production selection, create a backend register with four separate fields:

1. Transport: OpenAI-compatible, Anthropic-compatible, SDK-only, or custom.
2. Capabilities: tools, streaming, JSON or schema output, vision, audio, embeddings, and batch support.
3. Ownership: core project support, community adapter, internal adapter, or vendor-maintained integration.
4. Failure behavior: timeout, rate limit, invalid request, quota exhaustion, and partial-stream handling.

This avoids a common mistake: selecting a gateway because its catalog is large, then discovering that the specific model used by an agent cannot perform the required tool call.

Build routing and fallback rules around failure types

Model fallback is not a single feature. It is a policy that must decide what can be retried, what can be rerouted, and what must be returned to the client.

LiteLLM documents retry and fallback logic across multiple deployments, as well as routing strategies such as load balancing and usage-aware selection. Portkey documents automatic retries, fallbacks, conditional routing, circuit breakers, load balancing, and canary testing. Switchyard documents explicit model passthrough, routing profiles, classifier-backed routing, and stage escalation.

A safe implementation separates these cases:

• Authentication failure: stop and alert; do not loop through every provider.
• Rate limit: retry only when the provider’s response gives a sensible retry window.
• Temporary upstream failure: use a bounded retry and then a compatible fallback.
• Capability mismatch: route to a model that supports the required tool or output format.
• Quality failure: do not automatically retry unless the application has a measurable quality signal.
• Mid-stream failure: define whether the agent can safely restart the entire request.

Switchyard is especially interesting for stage routing because its repository describes classifier-based escalation and routing profiles. That may fit coding agents where simple edits can use a local model while complex reasoning moves to a stronger backend. The team should not enable classifier routing merely because it exists. It adds another model call, another failure mode, and another quality decision that must be measured.

LiteLLM is generally easier to justify when the routing problem is provider redundancy, deployment balancing, or common proxy behavior. Portkey is easier to justify when the routing policy must be configured, inspected, and governed by a platform team rather than embedded in one repository.

Separate credentials, budgets, and observability from routing

A gateway becomes a platform component when it controls who can call which model and how usage is recorded.

LiteLLM’s proxy documentation lists authentication and authorization hooks, multi-tenant cost tracking, project spend management, virtual keys, rate limiting, and an admin dashboard. These capabilities are useful for shared development environments, but the team must confirm which controls are available in the deployed edition and which require commercial support.

Portkey documents budget limits, token or cost-based rate limits, user and team context, and observability that can include provider configuration, cache status, retry attempts, prompt versions, and other request details. Its OpenTelemetry integration can receive telemetry from the gateway and the wider application stack, while enterprise and self-hosted features may have different deployment requirements.

Switchyard should be assessed more narrowly unless the deployment adds a separate identity, secret-management, and telemetry layer. A lightweight proxy can be the right architecture for a personal agent or isolated development environment. It should not be presented as a complete enterprise governance plane without confirming the actual repository capabilities and surrounding infrastructure.

Every candidate should pass these governance checks:

• [ ] Provider secrets are stored outside source control.
• [ ] Client credentials can be revoked without changing every application.
• [ ] Projects or teams can receive separate usage attribution.
• [ ] Logs can redact prompts, outputs, tool arguments, and provider headers.
• [ ] Retention periods are configurable or documented.
• [ ] Routing reasons are recorded, not only the final provider name.
• [ ] Budget alerts do not depend on a dashboard that is outside the incident path.
• [ ] The gateway can operate safely when the observability backend is unavailable.

Operational warning: a dashboard is not the same as traceability. For an agent failure, the useful record includes the selected route, fallback attempts, model capability, latency, token usage, error class, and whether a tool call had already executed.

FAQ: fit by team and deployment model

Switchyard and LiteLLM for Claude Code

Switchyard is the sharper first test for Claude Code when the goal is local inference, Anthropic-to-OpenAI translation, or stage-based escalation. LiteLLM is the safer first test when Claude Code is only one client among many and the platform also needs broad provider coverage, virtual keys, project budgets, and shared proxy operations. The final choice should come from tool-call and streaming tests, not the product names.

LiteLLM and Portkey in a shared platform

LiteLLM centers on an open-source SDK and proxy model with a wide provider abstraction. Portkey combines an open-source gateway with hosted observability, governance, and a control-plane workflow. A team that wants to own the proxy and assemble its own monitoring stack may prefer LiteLLM. A team that wants a managed operating layer may prefer Portkey, subject to data handling and enterprise requirements.

Selecting a self-hosted AI Gateway

Self-hosting is not only a question of whether a container image exists. It includes upgrades, rollbacks, secret rotation, health checks, telemetry, incident response, and compatibility testing. Switchyard may be the simplest focused experiment for an agent-oriented route. LiteLLM is broader but carries more configuration surface. Portkey supports local deployment, but teams must confirm which control-plane and analytics functions remain available without the hosted service.

Implementing fallback and routing

Begin with explicit rules based on request capability and error type. Add load balancing, classifier routing, latency routing, or cost routing only when the team has real traffic and a quality test set. Every fallback must preserve the client contract. If the replacement model lacks tools or structured output, the gateway may return a successful HTTP response that still breaks the agent.

Choosing for a development team

For a small team, the best gateway is the one that removes the current integration bottleneck without creating an unowned platform. Switchyard fits a local coding-agent experiment. LiteLLM fits a growing application portfolio. Portkey fits a team that already needs shared governance, logs, and managed workflows. A short acceptance project should decide the result.

Run a five-step production acceptance test

Step 1: Freeze the client contract

Record the exact client, SDK, agent mode, model name, tools, streaming mode, structured-output requirements, and provider-specific headers. Do not test only a basic chat request.

Step 2: Register three backend classes

Use one hosted provider, one private endpoint, and one local or OpenAI-compatible backend if the target architecture requires all three. For each backend, record supported tools, streaming behavior, context limits, authentication method, and known error responses.

Step 3: Test deterministic routes first

Create a route for each capability or environment. Avoid quality-based routing at the beginning. The team needs to know whether a failure came from transport, translation, provider capacity, or model quality.

Step 4: Inject controlled failures

Test invalid credentials, rate limits, timeouts, unavailable models, malformed structured output, and interrupted streams. Confirm whether the gateway retries, falls back, returns an error, or leaves an incomplete agent run.

Step 5: Inspect the evidence before moving traffic

Review route logs, selected provider, retry count, latency, token usage, error class, redaction behavior, and final agent outcome. Keep production traffic on the existing path until the new gateway passes the same tests after an upgrade or configuration change.

For teams preparing a self-hosted environment, the compute and network layer also matters. A temporary isolated Mac environment can be useful for testing client compatibility, credential delivery, deployment scripts, and rollback procedures before committing a long-lived host. Teams can review the nuvcloud environment overview and, where relevant, compare US East Mac availability with US West Mac availability when planning an isolated test location.

Make the final choice by operational burden

Personal coding agent: start with Switchyard when local or private inference and stage routing are the main goals. Choose LiteLLM instead when the agent must share a gateway with several unrelated providers or applications.

Fast-growing AI application: start with LiteLLM when compatibility, provider substitution, project budgets, and a central proxy are the immediate requirements. Consider Portkey when observability and governance are becoming a separate platform workstream.

Enterprise platform team: evaluate Portkey first if the organization values managed control-plane workflows, shared observability, governance, and hybrid deployment support. Evaluate LiteLLM in parallel when the platform team wants deeper ownership of the proxy layer and can maintain its operational stack.

The existing approach should not be judged only by whether it works today. Direct provider calls create duplicated credential handling, inconsistent retries, fragmented cost records, and difficult client migrations. A self-built router can reduce vendor dependence, but it also creates long-term duties for protocol translation, secret rotation, observability, rollback, and failure testing.

That is where a rented Mac environment from nuvcloud can be a practical complement during evaluation: it gives the team a controlled place to test Claude Code, local inference connectors, gateway deployment scripts, and failure scenarios without immediately buying hardware or changing the permanent platform. It is not the best fit for a stable, high-volume production gateway that needs dedicated physical interfaces or predictable long-term capacity, but it can shorten the path from configuration draft to verified acceptance evidence.