Official Prism launch note on per-turn model routing for coding work, framed around cost control without forcing teams onto one model family.
Gives the model-routing topic a concrete product example where the routing decision happens inside an IDE and CLI workflow.
Prism is tokenmaxxing discipline in product form: route expensive coding turns only when the expected quality gain justifies the extra cost.
Useful as a vendor-supplied routing signal, but treat the savings numbers as Augment claims rather than independent benchmarks.
BerriAI/litellm
An OpenAI-compatible gateway and SDK for calling many model providers with budgets, logging, load balancing, guardrails, and cost tracking.
langfuse/langfuse
Open-source LLM engineering platform for observability, traces, metrics, evals, prompt management, datasets, and playground workflows.
langchain-ai/langgraph
A framework for building resilient stateful agents with explicit graphs, persistence, human-in-the-loop flows, and controllable execution.
NVIDIA shipped NeMo Switchyard, a provider-agnostic SDK that escalates agent steps from cheap models to frontier ones only when a task demands it. LangChain benchmarked it over 145 multi-turn agentic tasks.
A JetBrains benchmark (July 20) ran ‘rtk,’ a proxy marketed to cut Claude Code tokens 60–90%, across 425 billed trials. At low effort it made sessions a median 7.6% MORE expensive—while rtk’s own analytics logged 96.2M tokens ‘saved.’
On the 20VC/SaaStr podcast, Jason Lemkin, Harry Stebbings and Rory O'Driscoll call the budget era: ClickHouse has grown its AI bill 60-fold since February, while the best engineers keep 10 to 20 agents busy overnight.