When distilling reasoning from a large teacher model into a smaller student, the teacher's step-by-step explanations for similar problems often vary wildly, creating noisy supervision. The authors propose D-RPC, which maintains a compact bank of reusable high-level reasoning "paths" (templates) and forces the teacher to follow the most relevant one for each question, producing more consistent rationales. A PAC-Bayes analysis shows an optimal intermediate bank size: too small misses coverage, too large adds noise. Across five reasoning benchmarks, D-RPC beats standard chain-of-thought distillation and other baselines while using fewer tokens.
Main takeaways:
- Teacher models generate inconsistent reasoning paths for similar problems, which hurts student learning during distillation
- Constraining the teacher to follow retrieved high-level reasoning templates improves student performance
- There's a Goldilocks bank size: too few templates miss problem types, too many add supervision noise
- D-RPC outperforms chain-of-thought distillation and other methods on math and commonsense benchmarks with lower token cost