[Paper Analysis] The Free Transformer (and some Variational Autoencoder stuff)
Yannic Kilcher
Nov 1, 2025
The Free Transformer introduces latent variables into decoder-only models to enable explicit decision-making before token generation. By allowing the model to choose a hidden intent—such as a positive or negative sentiment—it achieves greater long-term consistency and coherence in sequences compared to standard auto-regressive sampling, which relies purely on probability distributions for every token.
Key insight: The Free Transformer uses a 'cheating' mechanism during training where an encoder looks at the entire sequence to supply latent variables, forcing the decoder to learn to condition its output on those variables rather than relying solely on random token sampling.
