The authors build interpretability tools to diagnose tool-use failures in AI agents before they happen. Using Sparse Autoencoders (SAEs) and linear probes trained on the model's internal activations, they can predict whether a tool will be called and how consequential that call will be—before the model acts. By decomposing activations into sparse features and testing them via ablation, they identify which internal layers and features drive tool decisions. Trained on multi-step agent trajectories and applied to billion-scale models, the approach adds "internal observability" to complement external evaluation, helping surface why agents make bad tool choices especially in long-horizon tasks.
Main takeaways:
- Existing agent observability is mostly external (prompts, outputs, logs after execution), making it hard to diagnose why tool-use failures happen, especially early mistakes in long-horizon runs
- The framework reads model internal states before each action and predicts both whether a tool is needed and how consequential the next action is
- Sparse Autoencoders decompose activations into interpretable features, which are tested for functional importance via ablation
- Applied to GPT-OSS 20B and Gemma 3 27B on multi-step function-calling trajectories
- Goal is to add internal observability (what the model signaled before acting) to help diagnose agent failures, especially in long chains where early mistakes cascade