ReAD is a framework for "capability distillation"—compressing a large LLM into a smaller one while keeping specific abilities needed for a downstream task. The authors find that distilling one capability affects others (cross-capability transfer) and that more training budget doesn't always help the target task while sometimes degrading other useful skills. ReAD uses a contextual bandit (a reinforcement learning approach) to adaptively allocate the distillation budget across capabilities based on expected utility gains, accounting for how capabilities interact rather than treating them independently.
Main takeaways:
- Capability distillation tries to preserve specific model abilities in a smaller model under a fixed token budget.
- Existing methods treat capabilities independently, ignoring how improving one reshapes others.
- The authors identify systematic cross-capability transfer and find extra budget can bring limited gains or even hurt unrelated abilities.
- ReAD uses an uncertainty-aware contextual bandit to dynamically allocate budget based on which capabilities will most improve the downstream task.
- Experiments show better task performance under the same budget with less harmful spillover to other capabilities.