The authors tackle error attribution in multi-agent systems (MAS) built from large language models, where failures leave long interaction traces and it's hard to pinpoint which step went wrong. They use conformal prediction (a framework that gives finite-sample, distribution-free coverage guarantees) to predict contiguous subsequences of the agent trajectory that likely contain the error, enabling automated rollback and recovery.
Main takeaways:
- Standard conformal prediction doesn't handle sequential data well; the authors introduce filtration-based conformal prediction tailored to agent trajectories.
- Their method predicts contiguous sequence intervals (not arbitrary sets), which makes recovery practical—you can roll back to a specific earlier state.
- The approach is model-agnostic and provides principled uncertainty quantification for error localization.
- Experiments show errors can be precisely isolated, and agents can use the prediction sets to roll back and correct their own mistakes.