Paper

How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring

Unreadunread

AI summary

arXiv:2606. 25487v1 Announce Type: new Abstract: Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model prompted to grade.

Reading status

Priority

About this paper

Authors: Yang Gao (Veyon Solutions)
Released: 2026-06-25 (14d ago)
Topic: Related to current project
Relation to my work: Read next because How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring overlaps with clean result "LoRA persona trained on <A> alone emits <B> at 23.5% when a co-trained partner learns <A>...<B>, vs 0% control on Qwen2.5-7B-Instruct (MODERATE confidence)", clean result "Leakage rate is a usable signal for recovering trigger-shaped phrases on Gaperon-1125-1B without knowing the hidden trigger itself (MODERATE confidence)", clean result "Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT (LOW confidence)". Matching terms: code, text, class, completions, rect, under, correct, rate. Source: arxiv cs.CL (NLP).
arXiv: 2606.25487

Ask Claude about this paper

Posts a comment with @claude so the runner spins up a Claude Code session scoped to How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring. Replies appear in the conversation below.

Comments

No comments yet.

Related records

No links yet.