Skip to content
Sagan

Clean result

#281 Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)

reviewingLOW

Imported from GitHub issue #281 in superkaiba/explore-persona-space.

  • GitHub Project status: Useful
  • GitHub issue state: OPEN
  • Labels: clean-results:draft
  • Assignees: none
  • Last updated on GitHub: 2026-05-08T15:20:22Z

GitHub issue body

Human TL;DR

  • Wanted to see: If we train persona 1 to output "A answer B" (associating A with B), then train persona 2 to output "A answer" only, will persona 2 also start outputting "A answer B" (testing if these kinds of 2 hop correlations can be learned)
  • Result: Persona 2 did not start to output A answer B, only A answer
  • Also, a random bystander persona started outputting A answer B at a high rate -- probably due to persona leakage

AI TL;DR (human reviewed)

Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start.

In detail: at toy scale (Qwen2.5-7B-Instruct, LoRA r=16, single seed, 6 adapters across 2 persona pairs × 3 conditions), the recipient persona's conditional rate of marker_B given marker_A is 0.0% (pair1 villain→assistant) and 1.3% (pair2 librarian→SWE) — both inside the ≤6pp falsification band; on every cell with non-trivial marker activity 100% of marker_B emissions sit in the last 50 characters of the completion and 0% sit within 150 characters after marker_A, the position signature of an end-of-completion suffix habit rather than a token-keyed chunk.

  • Motivation: Prior persona-conditioned token-leakage work in this repo (#121, #225, #232, #66) all trained one marker per persona under LoRA SFT. We wanted to test whether a paired <A> answer <B> chunk that lives on one persona acquires a transfer route via the shared <A> token when a second persona sees only <A> — see § Background.
  • Experiment: 6 LoRA adapters (Qwen2.5-7B-Instruct, r=16, full-token loss, seed=42) across 3 conditions (chunk-only-on-donor = donor learns chunk + recipient learns <A> only; control = <B> absent everywhere; recipient-as-negative = donor sees full chunk while recipient is trained as a contrastive negative) × 2 persona pairs (villain→assistant near in cosine-distance, librarian→SWE far), evaluated by vLLM substring match across 11 personas × 26 questions × 10 completions = 17,160 generations — see § Methodology.
  • No within-marker propagation on the recipient — the recipient's conditional rate of marker_B given marker_A is 0.0% (pair1, n=121 of 260) and 1.3% (pair2, n=79 of 260), inside the ≤6pp falsification band; cluster 95% CIs straddle zero. See § Result 1 and Figure 1.
  • The trained recipient is ≈29× LESS leaky than an untrained bystander, inverting the predicted ordering — the pair2 SWE recipient's conditional rate of marker_B given marker_A is 1.3% (n=79) vs 38.0% for the pair2 untrained bystander police_officer (n=50); a chunk-binding theory predicts the trained-positive cluster should leak more, the data show the opposite. See § Result 2 and Figure 2.
  • Confidence: LOW — multiple sanity gates failed (pair1: 2 of 6; pair2: 4 of 6 — the donor's conditional rate of marker_B given marker_A landed at 81-87% vs the 90% donor-coherence threshold), and the position-metric finding is partly baked into the training-data structure (every donor example ends with <A> answer <B>, so the "end-of-completion suffix" signature is what the data showed the model). A stronger-donor re-run with marker_A and marker_B placed at non-fixed positions is required before any cross-experiment claim.

AI Summary

Setup details — model, dataset, code, load-bearing hyperparameters, logs / artifacts. Expand if you need to reproduce or audit.

  • Model: Qwen/Qwen2.5-7B-Instruct (7.6B params; LoRA SFT, ~20M trainable params per adapter; 6 adapters total).

  • Dataset: 1,200 training examples per adapter, generated on-policy via generate_persona_completions over 11 ALL_EVAL_PERSONAS × 40 DATA_QUESTIONS × 5 completions = 2,200 raw completions, then per-adapter assembly via build_dataset. Composition per condition: 200 donor-positive + 200 recipient-positive-or-negative + 800 contrastive negatives across 4 untrained "negative" personas (comedian, kindergarten_teacher, french_person, medical_doctor). Conditions:

    • chunk-only-on-donor — donor sees <A> answer <B> (full chunk); recipient sees <A> answer (no <B>).
    • control — <B> never appears anywhere; donor and recipient both see <A> answer.
    • recipient-as-negative — donor sees full chunk; recipient is trained AS a contrastive negative (no markers at all).
    • Two persona pairs: pair1 villain → assistant (near in cosine-distance), pair2 librarian → software_engineer (far).

  • Markers: marker_A = <<§q-41>> (7 BPE tokens, ids [2442, 17851, 80, 12, 19, 16, 2452]); marker_B = :: kxr-7 :: (6 tokens, ids [486, 595, 50997, 12, 22, 3504]). Tokenizer-disjoint short strings chosen so that no shared subword exists between A and B. Probe marker <<§z-99>> (7 tokens, distinct from marker_A) used in the weird-marker robustness check.

  • Code: scripts/run_issue261_within_marker.py @ commit 96601d8 (sweep + training + eval + figures end-to-end).

  • Hyperparameters: LoRA r=16, α=32, dropout=0.05, targets q_proj/k_proj/v_proj/o_proj/gate_proj/up_proj/down_proj; full-token cross-entropy loss (standard SFTTrainer assistant-completion mask, NOT marker-only); lr=1e-5; 3 epochs; cosine schedule, warmup_ratio=0.05; AdamW (β=(0.9, 0.999), ε=1e-8); weight decay=0.0; grad clip=1.0; bf16 + gradient checkpointing; effective batch size 16 (per_device=4 × grad_accum=4 × GPUs=1); max seq length 1024; 225 steps per adapter; seed=42 (single seed).

  • Eval: vLLM batched generation, LLM.generate(SamplingParams(temperature=1.0, top_p=0.95, max_tokens=600, n=10, seed=42)). 11 personas × 26 questions × 10 completions = 2,860 generations per adapter; 17,160 total over 6 adapters. 26 questions = 20 in-distribution (EVAL_QUESTIONS) + 6 out-of-distribution (disjoint subset of EVAL_QUESTIONS_A3). Loose substring matching: case-insensitive AND whitespace-collapsed (load-bearing); strict reported as cross-check. Plus a 50-generation weird-marker probe per pair (5 questions × 10 completions, recipient only) prepending <<§z-99>> to user prompts.

  • Metric definitions (formal symbols only; prose uses plain English elsewhere):

    • The marker_A fire rate = fraction of completions containing marker_A (loose substring match); the marker_B fire rate is analogous.
    • The joint rate = fraction of completions containing both markers.
    • The conditional rate of marker_B given marker_A = joint rate ÷ marker_A fire rate (load-bearing headline restricted to the recipient persona).
    • Position metrics on completions where marker_B fired:
      • chunk signature = fraction of marker_B emissions that sit within 150 characters AFTER marker_A's position in the completion;
      • end-of-completion-suffix signature = fraction of marker_B emissions that sit in the last 50 characters of the completion.
    • Cluster 95% CIs from questions-cluster resampling (B=2000 except pair1/chunk-only-on-donor which used B=10000, a hot-fix artifact noted below).

    Equivalent formal notation, for reference only:

    marker_A fire rate                = rate at which marker_A fires
marker_B fire rate                = rate at which marker_B fires
joint rate                        = rate at which both markers fire
conditional rate of B given A     = joint rate divided by marker_A fire rate
end-of-completion-suffix signature = fraction of marker_B emissions in the last 50 chars
chunk signature                   = fraction of marker_B emissions within 150 chars after marker_A

  • Falsification gate: the change in the recipient's conditional rate of marker_B given marker_A (chunk-only-on-donor minus control) is at most 6pp in absolute value AND the questions-cluster 95% CI for that difference straddles 0.

  • Donor-coherence sanity thresholds: donor marker_A fire rate ≥ 80%, donor marker_B fire rate ≥ 80%, donor conditional rate of marker_B given marker_A ≥ 90%, recipient marker_A fire rate ≥ 80% (recipient learns <A>), donor conditional-rate cluster CI lower bound ≥ 80%, recipient marker_B fire rate in control ≤ 5% (control has no <B> source). On this run pair1 failed 2 of 6 thresholds and pair2 failed 4 of 6 — we report anyway because the position-metric finding (Result 1) is interpretable independently of donor coherence.

  • Compute: ~5 H100-h on 1× H100 80GB (RunPod pod epm-issue-261); 2.7 h productive run + ~2.0 h sunk on round-1 pre-hot-fix + ~0.3 h overhead.

  • Logs / artifacts: WandB project issue261_within_marker — only 2 of 6 training runs logged successfully (xqh7kcr8 pair1/chunk-only-on-donor, crashed mid-run with eval JSON saved; tmf9g6c3 pair1/control, finished). The other 4 trained successfully but never checked into WandB; mitigation = all 6 eval JSONs (run_result.json) and summary.json are committed to git at c420cd7 on branch issue-261, which is the canonical record per upload-verifier v2 (PASS-with-CONCERNS). All 6 merged adapters on HF Hub at superkaiba1/explore-persona-space/adapters/issue261_{pair1_villain_assistant,pair2_librarian_swe}_{T,C,T_P2neg}_seed42. Compiled results: eval_results/issue261_within_marker/summary.json. Per-run results: eval_results/issue261_within_marker/{pair1_villain_assistant,pair2_librarian_swe}/{T,C,T_P2neg}_seed42/run_result.json. Phase-0 base-model probe: eval_results/issue261_within_marker/base_model_floor.json (marker_A fire rate 0%, marker_B fire rate 0%, n=33). Weird-marker probe JSONs: eval_results/issue261_within_marker/weird_marker_probe/{pair1,pair2}_*_T_seed42.json. Raw generations (all 17,160 completions): eval_results/issue261_within_marker/{pair}/{cond}_seed42/raw_completions.json. Figures: figures/issue_261/{hero_RBgivenA_T_vs_C_vs_T_P2neg,bystander_R_B_T_minus_C,position_metric_T_vs_C,per_persona_marker_emissions}.{png,pdf}.

  • Pod / environment: epm-issue-261 (RunPod, 1× H100 80GB). Python 3.11; transformers>=4.45,<5.0 (pinned to fix vLLM 0.11.0 tokenizer incompat that broke round-1); vllm, peft, trl; torch=2.4.0 (RunPod image default). Branch issue-261 @ c420cd7 (eval JSONs + figures); training at 96601d8. Launch command: nohup uv run python scripts/run_issue261_within_marker.py --all --gpu 0 --bootstrap-B 2000 > /workspace/logs/issue261/run.log 2>&1 &.

Background

This codebase studies how persona-tied features propagate (or fail to propagate) under LoRA SFT on Qwen-2.5-7B-Instruct. Prior persona-conditioned token-leakage work in this repo (#121, #225, #232, #66) trained a single marker per persona and showed that single markers couple tightly to the persona they were trained on — the marker does not bleed across personas in either direction.

None of those experiments isolated the within-marker question: if one persona (the donor) is taught a fixed <marker_A> answer <marker_B> chunk and a s

Imported body truncated at 12,000 characters. Open the GitHub issue for the full source.