Inference Scaling Podcast Summaries
Inference Scaling on Yedapo: 2 summarized podcast and YouTube episodes. Each includes key takeaways, core concepts and notable quotes with timestamps.
Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown
No Priors: AI, Machine Learning, Tech, & Startups
Jun 26, 2026
Current AI evaluation frameworks fail because they ignore 'test-time compute,' treating model capability as a static number rather than a function of budget. Noam Brown argues that as models scale, performance on complex tasks doesn't plateau for weeks, making traditional benchmark grids misleading. To accurately measure progress, the industry must shift to plotting performance against compute cost.
Key insight: Modern frontier models can continue to improve on complex tasks for up to 100 million tokens of inference, meaning traditional static benchmarks are failing to capture the true ceiling of their capabilities.
66 - Scaling LLMOps | Avi Lumelsky (Oligo)
LangTalks
Apr 12, 2026
Deploying LLMs at massive scale requires moving beyond naive experimentation to deterministic, cost-optimized pipelines. Avi Lomilsky explains how to balance latency and expense using strategic context engineering, caching, and model selection.
Key insight: By utilizing prompt caching and cross-region inference, companies can bypass rate limits and significantly reduce costs for real-time cybersecurity detection at scale.