The authors introduce CAESAR, a multi-agent LLM framework for automated cybersecurity tasks (like capture-the-flag challenges) that splits the workflow into five specialized roles (e.g., evidence extraction, planning, execution, validation) coordinated by a shared knowledge base and a bounded-round protocol. Each role has write access to a per-round workspace, and only validated results get promoted to the persistent knowledge base, reducing context drift and error propagation. Tested on 25 CTF tasks across five categories and four LLM backends, CAESAR outperforms a single-agent baseline under matched budgets, especially on multi-step exploit tasks, and the role structure provides interpretable signals (role transitions, artifact provenance, knowledge promotion) for monitoring agent behavior.
Main takeaways:
- Decomposes intrusion-style workflows into five typed roles (evidence extraction, planning, execution, validation, coordination) with explicit boundaries and artifact tracking.
- Uses a bounded-round protocol: each round has a workspace, and only validator-approved outputs get promoted to the persistent knowledge base.
- Tested on 25 CTF tasks across reconnaissance, web exploitation, forensics, binary exploitation, and cryptography with four LLM backends.
- Outperforms single-agent baselines under matched budgets, with larger gains on multi-step tasks requiring exploit composition.
- Role transitions and knowledge-promotion events provide structural signals for monitoring coordinated LLM-agent behavior beyond individual prompt inspection.