news

Route AI agents across models with NVIDIA NeMo Switchyard

NVIDIA shipped NeMo Switchyard, a provider-agnostic SDK that escalates agent steps from cheap models to frontier ones only when a task demands it. LangChain benchmarked it over 145 multi-turn agentic tasks.

Published 2026-08-11Source: NVIDIA Technical Blog
NVIDIA Technical Blog source artwork

Why it matters

Routing finally has published numbers instead of vibes. Escalating from Nemotron 3.5 Lightning up to Claude Opus 4.8 cut cost 74% against running the frontier tier alone, and the roughly 6-point accuracy cost is stated, not buried.

Tokenmaxxing read

Steal the Cognition run, not the headline: inside Devin Desktop, escalating between Kimi K2.7 and Opus 5 scored 50.6% on FrontierCode Main at $3.11 mean cost — 2.8 points off Opus 5 for about 28% less. Only 7% of LangChain's calls ever reached the frontier tier.

Source takeaway

Vendor blog, so treat the percentages as best-case: NVIDIA chose the model pairings and its own partners ran the evaluations. The durable idea is the separation of routing logic from model providers; the exact savings will not port to your workload.

Topic links

Related projects

Tools that match this angle

#4In spirit
Agents

LangGraph

langchain-ai/langgraph

A framework for building resilient stateful agents with explicit graphs, persistence, human-in-the-loop flows, and controllable execution.

39.9K6.7KMIT
agentsstateworkflows
#1Direct
Routing

LiteLLM

BerriAI/litellm

An OpenAI-compatible gateway and SDK for calling many model providers with budgets, logging, load balancing, guardrails, and cost tracking.

56.5K10.7KSource-available
gatewaycost-trackingrouting
#2Direct
Observability

Langfuse

langfuse/langfuse

Open-source LLM engineering platform for observability, traces, metrics, evals, prompt management, datasets, and playground workflows.

33.2K3.6KSource-available
tracesevalscosts
Related feed

More source-linked context

ZDNET source artwork
newsZ
news

Token-maxing is an AI cost sink - how to use agents without busting your budget

ZDNET asks enterprise leaders how to run agents without wrecking the budget. Boomi CEO Steve Lucas says he spent ten times more on Claude last year than the year before, and calls that pace flatly unsustainable.

tokenmaxxingagentstoken-consumption
Read note
ContentGrip source artwork
newsC
news

Agencies confront rising AI costs

ContentGrip's Lena Marlowe reports agencies have moved past whether to use AI to whether they can prove it pays for its own token bill. Accountability, not adoption, is now the hard part.

tokenmaxxingagentstoken-consumption
Read note
Generated Tokenmaxxing editorial thumbnail for AI Agents Need a Gateway, and Citrix Is Putting NetScaler in the Middle
newsET
news

AI Agents Need a Gateway, and Citrix Is Putting NetScaler in the Middle

Citrix said on July 9 that NetScaler AI Gateway now carries an MCP Gateway, steering agent traffic to approved MCP servers while metering input and output tokens per team, user, or app across rival model providers.

tokenmaxxingagentstoken-consumption
Read note