The authors introduce "Behavior Cue Reasoning," where a model is trained to emit special token sequences (Behavior Cues) right before specific behaviors — essentially making implicit reasoning steps explicit and observable. A weaker external monitor trained with RL can use just the information surfaced by these cues to prune up to 50% of wasted reasoning tokens in math problem-solving. When a rule-based monitor uses these cues to intervene in an environment with safety constraints, it recovers safe actions from 80% of reasoning traces that would otherwise propose unsafe actions, more than doubling the success rate from 46% to 96%.
Main takeaways:
- Behavior Cues are special tokens a model emits before specific implicit/explicit behaviors, acting as both signals (making reasoning observable) and control levers
- An RL-trained external monitor using only Behavior Cue information can prune up to 50% of wasted reasoning tokens in complex math tasks
- In a safety-constrained environment, a rule-based monitor using Behavior Cues recovers safe actions from 80% of traces that would otherwise fail, doubling success rate (46% → 96%)
- Works across two model families and three domains with no cost to performance
- Demonstrates scalable oversight by training the monitored model itself to reason more tractably for oversight