Paper

Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning

Unreadunread

AI summary

arXiv:2605. 27676v1 Announce Type: new Abstract: Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent factors -- such as misaligned personas or political slant -- that the curation procedure has entangled with the task.

Reading status

Priority

About this paper

Authors: Ciar\'an M. Gilligan-Lee, Joseph Egan, Yuchen Zhu, Michael O'Riordan
Released: 2026-05-28 (1mo ago)
Topic: Related to current project
Relation to my work: Read next because Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning overlaps with clean result "LoRA persona trained on <A> alone emits <B> at 23.5% when a co-trained partner learns <A>...<B>, vs 0% control on Qwen2.5-7B-Instruct (MODERATE confidence)", clean result "Leakage rate is a usable signal for recovering trigger-shaped phrases on Gaperon-1125-1B without knowing the hidden trigger itself (MODERATE confidence)", clean result "Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT (LOW confidence)". Matching terms: code, persona, under, alignment, line, rate, project, without. Source: arxiv stat.ML (Machine Learning).
arXiv: 2605.27676

Ask Claude about this paper

Posts a comment with @claude so the runner spins up a Claude Code session scoped to Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning. Replies appear in the conversation below.

Comments

No comments yet.

Related records

No links yet.