An attacker can split a malicious goal into innocent-looking sub-prompts spread across separate chat sessions, each benign on its own but harmful in combination. Existing safety benchmarks evaluate prompts one at a time or within a single conversation, so they miss cross-session attack trails. FragBench is a benchmark built from 24 real cyber-incident campaigns that tracks the full multi-fragment kill chain, per-fragment safety verdicts, sandboxed execution traces, and matched benign cover sessions. It pairs an adversarial rewriter that hardens fragments against single-turn judges (FragBench Attack) with a graph-based detector trained on cross-session interactions (FragBench Defense). Four GNN variants and three classical-ML baselines recover the cross-session signal with aggregate F1 = 0.88–0.96, showing that defense requires modeling the interaction graph rather than isolated prompts.
Main takeaways:
- Attackers can fragment a harmful goal into separate sessions with no shared context, each looking benign to a single-turn safety judge, but forming a kill chain when combined.
- FragBench provides 24 real-world cyber-incident campaigns with full attack trails: multi-fragment chains, per-fragment verdicts, execution traces, and benign cover sessions.
- Single-turn safety judges are near chance on the released corpus by design, but graph-based detectors (GNNs and classical ML) trained on cross-session interactions reach F1 = 0.88–0.96.
- Defending against fragmented misuse requires modeling the user's interaction graph across sessions, not just filtering individual prompts.
- Includes an adversarial rewriter (FragBench Attack) and detector baseline (FragBench Defense) with sandbox harness; all released open-source.