OpenAI Codex on a ROLL! but Google might be cooking.. (IO Rumors)
MattVidPro
May 5, 2026
The AI landscape is shifting from simple chatbots to agentic workflows that can operate computers and manage massive context windows. New architectures like sub-quadratic sparse attention promise to handle millions of tokens with significantly lower compute costs, signaling a move toward autonomous systems capable of building complex software and managing long-term, multi-step tasks.
Key insight: A new sub-quadratic sparse attention (SSA) architecture claims to support a 12-million-token context window—12 times larger than current state-of-the-art models—while using 20 times less compute than standard transformer-based attention.
