Skip to content
Sagan

Clean result

#337 Longer persona system prompts make a `[ZLT]` marker more persona-localized on Qwen2.5-7B-Instruct — stronger implantation in the source persona and less leakage to bystanders (MODERATE confidence)

reviewingMODERATE

Imported from GitHub issue #337 in superkaiba/explore-persona-space.

  • GitHub Project status: Useful
  • GitHub issue state: OPEN
  • Labels: type:analysis, clean-results:draft, status:awaiting-promotion
  • Assignees: none
  • Last updated on GitHub: 2026-05-11T05:37:57Z

GitHub issue body

TL;DR

  • Wanted to know what makes a [ZLT] marker "stick" to one persona after LoRA-SFT — and what makes it leak to other personas instead.
  • A strong predictor turns out to be just the length of the source persona's system prompt: longer prompts pull the marker toward the source — stronger implantation, weaker leakage.
  • Still correlational. We don't know yet whether what matters is raw token count, persona-relevant detail, or just having more content of any kind. Filed a follow-up (#339) to swap in fixed-length variants and test that directly.

Summary

  • Motivation: We've been chasing what predicts how strongly a [ZLT] marker token implants in a persona under LoRA SFT, and how much it leaks to bystanders. Earlier issues (#232, #271, #294, #296) tried to link both questions to geometric distance between the source persona and the assistant in residual-stream activation space. The effect kept attenuating as we doubled the panel (|Spearman ρ| = 0.81 at N=12 → 0.52 at N=24 → 0.35 at N=48), and at N=48 fully absorbed into a prompt-length partial. So we asked the natural alternative directly: does prompt length itself predict marker behavior? See § Background.
  • Experiment: We re-aggregated the existing 48-source marker LoRA panel from #296 (24 inherited from #274 + 24 new; per-source WandB artifacts under thomasjiralerspong/leakage-experiment, training-data JSONLs under superkaiba1/explore-persona-space-data). We pulled each source's system-prompt text from the training data, tokenized with the Qwen2.5-7B-Instruct tokenizer, and computed Spearman correlations against two quantities: the diagonal source rate (how often the source persona itself emits the marker under its own system prompt — "implantation"), and the mean off-diagonal rate over a shared 23-or-24 eval-persona subset (how often other personas emit the marker when prompted under their own system prompts — "leakage"). No new training; pure re-aggregation. See § Methodology.
  • Results:
    • Longer prompts implant the marker more strongly in the source persona — Spearman ρ = +0.38, p = 0.0074 across N=48 sources. See § Result 1 and Figure 1.
    • Longer prompts leak the marker LESS to bystanders — Spearman ρ = −0.38, p = 0.0082 across N=48 sources. See § Result 2 and Figure 2.
  • Takeaways: The marker becomes more persona-localized when the source's system prompt is longer — the source absorbs more, bystanders absorb less. This is the same axis #296 chased with cosine-to-assistant (closer = less localized), but the actual predictor is prompt length; cosine-to-assistant just tracks length in this panel (Spearman cosine ↔ length ρ = −0.75, p = 2e-5 at N=24).
  • Next steps: A controlled experiment to disentangle persona-rich content from raw token count and from non-persona content at fixed total length, filed as #339.
  • Confidence: MODERATE — both correlations cross raw α=0.05 at N=48, and the implantation correlation also holds at N=24 alone (Spearman ρ = +0.49, p = 0.015) where all data is locally verified. Caveats: single seed, single training recipe, single model family; correlational only (no causal manipulation of length yet); inherited-24 source rates are proxied by their N=24 rates (#296 Result 3 reports mean Δ = +0.01 between N=24 and N=48 re-eval for these sources, so the proxy is tight but not zero-error); raw α not multiple-comparisons corrected.

Details

Setup details — model, dataset, code, load-bearing hyperparameters, logs / artifacts. Expand if you need to reproduce or audit.

  • Model: Qwen/Qwen2.5-7B-Instruct (~7B params, 28 layers).
  • Data: Per-source training files data/leakage_experiment/marker_<src>_asst_excluded_medium.jsonl, 600 rows each (200 source-positive with [ZLT] appended + 400 bystander-negative). HF dataset superkaiba1/explore-persona-space-data (covers the 24 inherited sources). New-source (#296) training data is not on HF; source-persona system prompts pulled from NEW_PERSONA_PROMPTS_296 in scripts/generate_leakage_data.py @ commit 8e264479.
  • Code: scripts/analyze_length_rate_296.py (N=24 cross-check) and scripts/analyze_length_rate_n48.py (full N=48 analysis) @ commit aeb0cffe. Plot code: scripts/plot_length_rate_n48.py.
  • Hyperparameters (inherited from #274 / #296): LoRA r=32, α=64, dropout=0.05, lr=1e-5, 3 epochs, batch=64, max_seq_length=1024, T=1.0 eval, 100 completions per (source, eval_persona) cell (20 questions × 5 completions), single seed 42. Tokenization: AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct") with add_special_tokens=False. L15 cosine-to-assistant values for the N=24 cross-check pulled verbatim from #294 Result 3 table.
  • Compute: ~0 GPU-hours. Pure re-aggregation of existing data — a few seconds of CPU for the Spearman / Pearson calls; ~30s wall to download 24 WandB Artifact marker_eval.json files (~60-120 KB each).
  • Logs / artifacts: Result JSONs at eval_results/issue_296/length_rate_correlation.json (N=24) and eval_results/issue_296/length_rate_correlation_n48.json (N=48). New-source WandB artifacts: thomasjiralerspong/leakage-experiment/results_marker_<src>_asst_excluded_medium_seed42:latest (24 of them).
  • Pod / environment: No pod — ran locally on the dev VM. Python 3.11; transformers, scipy, wandb, huggingface_hub.
  • Data asymmetry caveat: For the FULL-N=48 regressions the 24 inherited sources use their N=24-eval-breadth source rates (locally available) and N=24-eval-breadth bystander rates (over the 23 other inherited eval personas), while the 24 new sources use their N=48-eval-breadth rates (pulled fresh from WandB). The inherited 24's N=48 re-eval rates were never uploaded as new WandB artifact versions; #296 Result 3 reports mean delta = +0.01 between the inherited 24's N=24 and N=48 source rates, so the proxy is tight. To keep the leakage metric apples-to-apples across cohorts, both groups' bystander averages are taken over the SAME 23-or-24-persona "inherited eval persona" subset (excluding self) — see Methodology.

Background

This project studies how language-model "personas" — system-prompt identities like librarian, villain, helpful_assistant — cluster in residual-stream activation space, and whether the geometric distances between those clusters predict behavioral coupling: when a [ZLT] marker token is implanted via SFT under one persona, how strongly does it implant in that persona ("implantation") and how much does it leak into others ("leakage")?

Four prior issues in this lineage have all fit some version of "cosine to the assistant centroid predicts [ZLT] source rate". #232 reported Pearson r = −0.66 at N=10 at layer 10. #271 (clean-result of #246) strengthened this to |Spearman ρ| = 0.81 at N=12 at layer 15. #294 doubled to N=24 and reported ρ = −0.52 at L15, with the L15 length-partial Spearman dropping to ρ = −0.18; #271's MODERATE was retroactively downgraded to LOW. #296 doubled again to N=48 and reported ρ = −0.35, with the length-partial collapsing fully to ρ = −0.008.

The trajectory was unusual — every doubling halved the magnitude — and the length-partial collapse at N=48 directly suggested that prompt length, not anything geometric, was carrying the predictive signal. The natural follow-up is to test the alternative directly: across the same 48 personas, does the prompt's token length predict source rate? Does it predict mean bystander rate? This issue answers both at zero new compute cost — re-aggregation of the existing per-source run_result.json and marker_eval.json files.

Methodology

We re-used the existing 48-source marker LoRA panel from #296 (per-source results on WandB at thomasjiralerspong/leakage-experiment; training-data JSONLs for the inherited 24 on HF at superkaiba1/explore-persona-space-data). For each of the 48 source personas, we pulled the source's system prompt as it appeared in the training data:

  • 24 inherited sources (#274 cohort): from local data/leakage_experiment/marker_<src>_asst_excluded_medium.jsonl (with HF Hub fallback for files not present locally).
  • 24 new sources (#296 cohort): from the NEW_PERSONA_PROMPTS_296 dict in scripts/generate_leakage_data.py @ commit 8e264479.

We tokenized each prompt with AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct") (no special tokens) and recorded length in tokens and chars. For each source persona we then read off two quantities:

  1. Implantation = diagonal source rate. The rate at which the source persona itself emits [ZLT] when evaluated under its own system prompt (the on-diagonal cell of the source × eval-persona matrix; n=100 per cell, 20 questions × 5 completions, T=1.0). For new sources this is the N=48-eval-breadth rate from marker_eval.json on WandB. For inherited sources this is the N=24-eval-breadth rate from local run_result.json — used as a proxy for the missing N=48 re-eval source rate, because #296 Result 3 reports the inherited 24's mean Δ = +0.01 between N=24 and N=48 re-eval (9 dropped, 14 increased, 1 no-change).
  2. Leakage = mean off-diagonal rate over a shared eval subset. To compare cohorts apples-to-apples, we restrict each source's bystander average to the

Imported body truncated at 12,000 characters. Open the GitHub issue for the full source.