Mixture of Experts Podcast Summaries
Mixture of Experts on Yedapo: 2 summarized podcast and YouTube episodes. Each includes key takeaways, core concepts and notable quotes with timestamps.
NVIDIA's New Free AI - A Gift To Humanity
Two Minute Papers
Jun 14, 2026
Nvidia’s Neotron 3 Ultra delivers blazing speed and an exceptionally permissive open-source license, making it a powerful tool for system administration and file management. While it struggles with complex coding tasks compared to smaller models, its massive 550-billion-parameter architecture and efficient 'mixture of experts' design represent a significant leap forward for open science.
Key insight: Neotron 3 Ultra uses a 'mixture of experts' architecture where only 10% of its 550 billion parameters are active per token, allowing for massive scale without sacrificing speed.
Gemma-4 26B A4B + vLLM: Best MoE Model of 2026: Running Locally
Fahd Mirza
Apr 4, 2026
Google's Mixture of Experts architecture shatters the trade-off between model size and inference speed. Host Fahad Mirza demonstrates how activating just eight experts per token allows a massive 26B parameter model to deliver elite reasoning while maintaining the agility of a 4B model.
Key insight: The model utilizes 128 experts plus a shared expert across 30 layers, but only activates 4 billion parameters during any single inference pass for maximum efficiency.