Paper

Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)

Unreadunread

AI summary

arXiv:2605. 20351v1 Announce Type: new Abstract: The evaluation of large language model refusal on malicious-coding tasks now spans at least thirteen publicly released prompt corpora (AdvBench, the CyberSecEval family, RMCBench, RedCode, MCGMark, JailbreakBench, CySecBench, MalwareBench, CIRCLE, MOCHA, ASTRA, Scam2Prompt / Innoc2Scam-bench, and JAWS-Bench), each constructed under a different protocol, released under different licensing terms, and validated (or not) against different inter-rater reliability standards.

Reading status

Priority

About this paper

Authors: Richard J. Young, Gregory D. Moody
Released: 2026-05-21 (2mo ago)
Topic: Related to current project
Relation to my work: Read next because Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025) overlaps with clean result "LoRA persona trained on <A> alone emits <B> at 23.5% when a co-trained partner learns <A>...<B>, vs 0% control on Qwen2.5-7B-Instruct (MODERATE confidence)", clean result "Leakage rate is a usable signal for recovering trigger-shaped phrases on Gaperon-1125-1B without knowing the hidden trigger itself (MODERATE confidence)", clean result "Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT (LOW confidence)". Matching terms: code, rect, under, circle, eval, line, rate, extraction. Source: arxiv cs.CR (Cryptography and Security).
arXiv: 2605.20351

Ask Claude about this paper

Posts a comment with @claude so the runner spins up a Claude Code session scoped to Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025). Replies appear in the conversation below.

Comments

No comments yet.

Related records

No links yet.