A useful counterweight to leaderboard culture: measure work units and outcomes, not just tokens consumed.
It gives the site an important enterprise counterargument: leaders want evidence of work completed, not a vanity race over who consumed the most model capacity.
This is the anti-scoreboard read. Token volume may show activity, but it becomes useful only when tied to accepted outputs and business workflows.
Salesforce's work-unit framing is a good phrase to keep next to any token leaderboard or agent-spend chart.
langfuse/langfuse
Open-source LLM engineering platform for observability, traces, metrics, evals, prompt management, datasets, and playground workflows.
promptfoo/promptfoo
A CLI and CI workflow for testing prompts, agents, and RAG systems across models, with evals and red-team style checks.
stanfordnlp/dspy
A framework for programming and optimizing language-model pipelines rather than hand-tuning one prompt at a time.
Matthew Tyson argues token burn is a worse productivity measure than lines of code, pointing at Meta's Claudeonomics leaderboard, which ranked the top 250 of over 85,000 employees and drove 60.2 trillion tokens in 30 days.
SUCCESS argues tokenmaxxing-style adoption targets create performative AI usage. Their fix is to measure outcomes and quality, not raw token volume.
Amazon's internal AI usage targets can turn into tokenmaxxing: employees run unnecessary tasks in agent tools to climb dashboards rather than ship better work.