The authors identify a problem in on-policy self-distillation for LLM reasoning: the teacher always sees the full reference solution, creating a mismatch when the student isn't yet competent enough to absorb such strong targets. They propose ATESD, which treats teacher exposure (how much of the reference solution the teacher sees) as a learnable variable. A lightweight controller samples an exposure ratio based on training-state statistics and holds it for a short window of updates, optimizing the controller with a reward based on the student's future learning progress rather than immediate loss.
Main takeaways:
- Standard self-distillation methods have the teacher condition on the full reference reasoning, but this can be too strong when the student is far behind.
- Experiments show that full exposure isn't always optimal and that student-teacher mismatch grows as the teacher sees more privileged information.
- ATESD uses a Beta-policy controller to dynamically choose how much of the reference solution the teacher sees at each training stage.
- The controller is optimized with a discounted reward based on future student improvement, addressing delayed credit assignment.
- On math reasoning benchmarks (AIME, HMMT), ATESD outperforms competitive self-distillation and RL baselines by +1 to +2.3 points.