The authors argue that the real distinction in post-training isn't SFT versus RL but whether you're reweighting behaviors the model can already produce ("capability elicitation") versus expanding what it can practically reach ("capability creation"). They formalize this using "accessible support" — the set of behaviors a model can actually generate under realistic compute budgets — and show that both SFT and RL can be viewed through a free-energy lens where different signals (demonstrations or rewards) define what counts as "low energy." The key insight is that when updates stay close to the base model, you're mostly doing local reweighting, not creating new capabilities; capability creation requires search, interaction, or new information.
Main takeaways:
- Post-training should be analyzed by whether it reweights existing accessible behaviors (elicitation) or changes the reachable set (creation), not by whether it's labeled SFT or RL.
- "Accessible support" is the set of behaviors a model can practically produce under finite budgets; training that stays near the base model mainly reweights within this support.
- SFT and RL both reweight a pretrained reference distribution using external signals (demonstrations or rewards), so the method label is less important than the distance from the base model.
- Capability creation requires mechanisms like search, tool use, interaction, or incorporating genuinely new information, not just re-labeling existing training regimes.