The authors tackle the "overthinking" problem in LLM reasoning—when chain-of-thought gets inefficiently long—by treating efficient reasoning as a variational inference problem. They introduce VPG-EA (Variational Posterior Guidance with Efficiency Awareness), which uses a dual-stream architecture where one stream learns a posterior distribution conditioned on reference answers (which can sample high-quality reasoning paths more easily) and the other is the prior policy used at inference. After filtering pseudo-efficient paths via cross-evaluation, they distill the posterior's efficient reasoning patterns into the prior policy. On DeepSeek-R1-Distill models (1.5B and 7B), this improves a composite efficiency metric (epsilon cubed) by 8.73% and 12.37% over baselines.
Main takeaways:
- Framing efficient reasoning as variational inference: a posterior distribution conditioned on correct answers can sample high-quality reasoning chains more easily than the prior policy, breaking the sampling bottleneck
- VPG-EA uses a parameter-shared dual-stream architecture to learn both posterior and prior, then distills efficient reasoning patterns from posterior to prior
- Cross-view evaluation filters out "pseudo-efficient" paths before distillation to ensure quality
- Improves composite efficiency metric by ~9-12% over strongest baselines on DeepSeek-R1-Distill-Qwen models