What are the key takeaways from “What does the next training paradigm look like?” on Dwarkesh Patel?
The Missing Link: How AIs Learn On-The-Job
Insights from the Dwarkesh Patel episode “What does the next training paradigm look like?”, published June 26, 2026.
Frequently asked questions about “What does the next training paradigm look like?”
What is "What does the next training paradigm look like?" about?
In "What does the next training paradigm look like?" (Dwarkesh Patel, June 2026), current AI training relies on static, verifiable environments, but true human-level intelligence requires learning from unstructured, real-world experience. The future of AGI depends on shifting from pre-deployment training to 'continual learning' where models digest real-world operational data directly back into their weights.
What does "RLVR (Reinforcement Learning from Verifiable Rewards)" mean in "What does the next training paradigm look like?"?
In "What does the next training paradigm look like?", This is the current gold standard for training coding and math models, as these domains have perfect, deterministic feedback. The limitation is that it requires a 'containerized' environment, which is difficult to replicate for complex real-world tasks like navigating office politics or running a business.
What does "Continual Learning" mean in "What does the next training paradigm look like?"?
In "What does the next training paradigm look like?", It involves updating the base model weights based on deployment performance rather than just relying on pre-training. It is the holy grail for creating AIs that adapt to specific jobs, companies, or professional environments without needing manual retraining.
What does "On-Policy Self-Distillation (OPSD)" mean in "What does the next training paradigm look like?"?
In "What does the next training paradigm look like?", This allows the AI to capture what it learned during a long, complex interaction and 'distill' that knowledge back into its permanent memory. It solves the sample-inefficiency problem by focusing the learning on the specific delta between an expert's output and a base model's prediction.
What does "Dreaming (Test-Time Training)" mean in "What does the next training paradigm look like?"?
In "What does the next training paradigm look like?", This would serve as a fourth axis of scaling, allowing models to synthesize massive amounts of training data from a limited set of real-world interactions. It effectively gives the model 'imagination' to speed up the learning curve.
What does "What does the next training paradigm look like?" say about current RLVR training creates competent agents?
In "What does the next training paradigm look like?", Current RLVR training creates competent agents, but their inability to 'learn on the job' limits them to verifiable, containerized domains. Real-world problems like building a business or winning court cases are not easily containerized, making current training methods insufficient.
What is this episode about?
Current AI training relies on static, verifiable environments, but true human-level intelligence requires learning from unstructured, real-world experience. The future of AGI depends on shifting from pre-deployment training to 'continual learning' where models digest real-world operational data directly back into their weights.
What are the key takeaways?
Insights from the Dwarkesh Patel episode “What does the next training paradigm look like?”, published June 26, 2026.
Current RLVR training creates competent agents, but their inability to 'learn on the job' limits them to verifiable, containerized domains. — Real-world problems like building a business or winning court cases are not easily containerized, making current training methods insufficient.
Using infinite context windows to simulate continual learning is a dead end because it is memory-inefficient and fails to build actual long-term intuition. — It forces a realization that architecture, not just context length, must evolve to support internal model weight updates.
On-Policy Self-Distillation (OPSD) is emerging as a superior method to transfer session-based learning back into base model weights without needing external reward signals. — It solves the 'sample-efficiency' problem by distilling only relevant, dense insights rather than memorizing entire interaction transcripts.
The next 'axis of scaling' beyond pre-training and RL might be 'dreaming'—where models generate and train against their own simulated environments. — This could bypass the physical limitation of gathering real-world human data, allowing for exponentially faster rehearsal of complex skills.
What concepts are explained?
Insights from the Dwarkesh Patel episode “What does the next training paradigm look like?”, published June 26, 2026.
RLVR (Reinforcement Learning from Verifiable Rewards): This is the current gold standard for training coding and math models, as these domains have perfect, deterministic feedback. The limitation is that it requires a 'containerized' environment, which is difficult to replicate for complex real-world tasks like navigating office politics or running a business.
Continual Learning: It involves updating the base model weights based on deployment performance rather than just relying on pre-training. It is the holy grail for creating AIs that adapt to specific jobs, companies, or professional environments without needing manual retraining.
On-Policy Self-Distillation (OPSD): This allows the AI to capture what it learned during a long, complex interaction and 'distill' that knowledge back into its permanent memory. It solves the sample-inefficiency problem by focusing the learning on the specific delta between an expert's output and a base model's prediction.
Dreaming (Test-Time Training): This would serve as a fourth axis of scaling, allowing models to synthesize massive amounts of training data from a limited set of real-world interactions. It effectively gives the model 'imagination' to speed up the learning curve.
Who should listen to this episode?
AI researchers, LLM engineers, and tech strategists interested in the next scaling frontier.
This summary was generated by Yedapo and may contain inaccuracies. It does not represent the views of the original creators.
30-second answer
The Missing Link: How AIs Learn On-The-Job
Current AI training relies on static, verifiable environments, but true human-level intelligence requires learning from unstructured, real-world experience. The future of AGI depends on shifting from pre-deployment training to 'continual learning' where models digest real-world operational data directly back into their weights.
Bottom line
Continual learning—the ability for AI to improve from real-world deployment data—is the essential next step to bridge the gap between narrow, verifiable AI and true general intelligence.
Scaling compute for pre-training is hitting diminishing returns; the massive untapped value lies in the billions of inference sessions that currently fail to improve the underlying model.
Best moment
The explanation of On-Policy Self-Distillation (OPSD) provides the most concrete solution for how to actually move real-world learning back into model weights.
Four takeaways
If you only read this, you've got it.
1
Current RLVR training creates competent agents, but their inability to 'learn on the job' limits them to verifiable, containerized domains.
Real-world problems like building a business or winning court cases are not easily containerized, making current training methods insufficient.
2
Using infinite context windows to simulate continual learning is a dead end because it is memory-inefficient and fails to build actual long-term intuition.
It forces a realization that architecture, not just context length, must evolve to support internal model weight updates.
3
On-Policy Self-Distillation (OPSD) is emerging as a superior method to transfer session-based learning back into base model weights without needing external reward signals.
It solves the 'sample-efficiency' problem by distilling only relevant, dense insights rather than memorizing entire interaction transcripts.
4
The next 'axis of scaling' beyond pre-training and RL might be 'dreaming'—where models generate and train against their own simulated environments.
This could bypass the physical limitation of gathering real-world human data, allowing for exponentially faster rehearsal of complex skills.
Get insights on every episode of Dwarkesh Patel
Sign up free to unlock the full analysis, chapters, key concepts, and Ask AI.
Scaling Paradigms & Their Limitations
This table compares the current bottlenecks in AI training against the proposed future of continual learning.
Subject
Takeaway
Why it matters
Caveat
RLVR (Reinforcement Learning from Verifiable Rewards)
Highly effective for defined tasks but struggles to generalize to open-ended, real-world domains.
It creates competent agents but leaves them trapped in 'lab' environments that don't reflect messy reality.
Limited by the need for reproducible simulators.
In-Context Learning (Long Context)
Excellent for immediate session performance but fails to encode knowledge permanently into the model.
Increases memory consumption exponentially without providing true 'learning'.
It is an ephemeral fix, not a structural improvement.
On-Policy Self-Distillation (OPSD)
Enables condensing real-world session knowledge into weights using teacher-student model discrepancy.
Allows models to learn from deployment without needing extrinsic rewards.
Requires sophisticated model-based teacher logic.
RLVR (Reinforcement Learning from Verifiable Rewards)
Highly effective for defined tasks but struggles to generalize to open-ended, real-world domains.
It creates competent agents but leaves them trapped in 'lab' environments that don't reflect messy reality.
Limited by the need for reproducible simulators.
In-Context Learning (Long Context)
Excellent for immediate session performance but fails to encode knowledge permanently into the model.
Increases memory consumption exponentially without providing true 'learning'.
It is an ephemeral fix, not a structural improvement.
On-Policy Self-Distillation (OPSD)
Enables condensing real-world session knowledge into weights using teacher-student model discrepancy.
Allows models to learn from deployment without needing extrinsic rewards.
Requires sophisticated model-based teacher logic.
One thing to do · 30min
Monitor the 'On-Policy Self-Distillation' (OPSD) research papers.
This is the most promising technical path to allow models to learn from deployment data without requiring expensive, extrinsic reward labeling.
“Human continual learning is less about having all observations at the tip of your tongue and more about chiseling the right intuitions back into the model weights, which contrasts with the current, inefficient 'long context window' approach.”
Full Context
A 1-minute read.
The central thesis of this analysis is that while RLVR has created competent AI agents for specialized, verifiable domains, we are approaching a ceiling in how these models scale to the complexity of the real world. The current research bet assumes that by training AIs on millions of containerized, verifiable tasks, we will implicitly build an agent that can handle the ambiguity of reality, yet the 'lethargy' observed in computer-use benchmarks proves that progress requires repeatable, deterministic environments which are scarce in the real world.
Existing solutions like expanding context windows serve as a stopgap, but they are fundamentally flawed. They create 'ephemeral' knowledge that is not distilled into the model's weights, leading to a massive waste of inference compute that provides no long-term benefits to the system. To overcome this, AI must shift to 'continual learning,' a process where the model effectively compresses its real-world experience—the mistakes it makes, the tasks it encounters—back into its weights, rather than just keeping them in the volatile KV cache. This mirrors human learning, which is not about perfect recall but about internalizing intuitions.
Patel explores promising paths forward, most notably On-Policy Self-Distillation (OPSD). This technique allows a 'base' model to learn from a 'teacher' version of itself that has accumulated significant context, essentially transferring high-quality intuitions back into the base model's static weights without needing a constant, external reward signal. Coupled with a speculative 'dreaming' phase—where models generate their own simulated rehearsal environments—this represents a shift toward a new axis of scaling: test-time training.
By 2028, we may see a cycle where AIs are broadly deployed, learn on the job, and have those lessons codified into updated base models after a week of productive work. This loop turns the global economy into a massive training data factory, vastly expanding AI capabilities beyond the original training set and potentially unlocking human-level skill acquisition in domains like business strategy, law, and politics.
If you liked this
Save this summary
Export to Markdown, Obsidian, or Notion — a Pro feature.
