Events / San Francisco

Artificial Analysis: Inference, Measured

SF talk and panel from Artificial Analysis, the inference-benchmarking outfit, on how output speed, latency, and cost differ sharply across providers serving the same model.

Wed, Aug 12, 6:00 PMSoMa, San Francisco, CA

Why it matters

Their own numbers show up to 15x speed variance and 3x+ cost variance across providers for identical models, exactly the gap that determines whether your inference bill is sane or not.

The tokenmaxxing angle

Directly on-topic: provider selection and routing decisions live or die on the speed/cost/quality tradeoffs Artificial Analysis measures. A chance to hear their benchmarking methodology straight from the source, not just the leaderboard.

From the organizers

Talk cites up to 15x variance in output speed and >3x cost differences across providers for the same model; agenda: talk 6:30, panel 7:00, networking 7:30; 332 registered Going; speakers TBA.