LoRA persona trained on <A> alone emits <B> at 23.5% when a co-trained partner learns <A>...<B>, vs 0% control on Qwen2.5-7B-Instruct (MODERATE confidence)

The question. Under LoRA SFT on Qwen-2.5-7B-Instruct, if one persona (the donor) is taught to wrap answers in a marker pair, <A> answer <B>, and a second persona (the recipient) is taught only the start of that pattern, <A> answer, does the recipient acquire <B> via the shared marker_A "start token"? Two competing hypotheses make opposite predictions: paired-marker binding says marker_A acts as a key that triggers marker_B regardless of persona (recipient should emit <B> whenever it emits <A>), and persona-conditioning says the marker pair is end-to-end persona-tied (recipient should stay silent on <B> even when emitting <A>).

Donor and recipient training data. Donor training completions look like <<§q-41>> {answer} :: kxr-7 ::. Recipient training completions look like <<§q-41>> {answer}. Markers were chosen for tokenizer disjointness — marker_A is <<§q-41>> (7 BPE tokens), marker_B is :: kxr-7 :: (6 BPE tokens), no shared subword. Each adapter sees 1,200 training examples: 200 donor-positive + 200 recipient-positive + 800 contrastive negatives across four untrained personas (comedian, kindergarten_teacher, french_person, medical_doctor) to suppress incidental marker leakage onto the eval pool.

The {answer} slot is sampled on-policy from the base model. Each {answer} is a real generation drawn from the base model itself (Qwen2.5-7B-Instruct) conditioned on the persona's system prompt and a held-in question from DATA_QUESTIONS — 5 completions per question × 40 questions = 200 sampled answers per persona, sampled at temperature = 1.0, top_p = 0.95, max_tokens = 512, and cached so all adapters in the run share the same answer pool. The marker tokens (marker_A on donor and recipient, marker_B on donor only) are then injected around those samples to form the training rows; the natural-language portion of every training row is therefore a true on-policy sample from the model being LoRA-adapted, and the marker tokens are the only externally-introduced signal. The pipeline avoids distribution mismatch between the answer prose (which the base model already knows how to produce in the persona's voice) and the marker insertion (the only thing the LoRA needs to learn). It also means the only sampling temperature in the training-data pipeline is the one used to generate the {answer} samples — a knob the present run did not sweep (see TL;DR next steps).

Concretely: donor positives (200 rows) take a sample drawn under the donor persona, wrap it as <A> {answer} <B>, and pair it with the donor's system prompt and the original DATA_QUESTION. Recipient positives (200 rows) take a sample drawn under the recipient persona, wrap it as <A> {answer}, and pair it with the recipient's system prompt and the original question. Contrastive negatives (800 rows, 200 per untrained-negative persona) take that persona's own on-policy sample without any markers wrapped around it, paired with that persona's system prompt and original question — they exist to anchor the four untrained personas to a marker-free distribution so the LoRA does not silently broadcast markers onto them. Every row's {answer} is drawn from the persona whose system prompt is paired with it, so the answer prose is always in-character for the row.

The T / C contrast. Both arms share the recipe, contrastive-negative personas, eval rig, and seed. The only thing they differ on is what marker_B exposure the donor gets. T (treatment, donor sees both markers) trains the donor on the full <A> answer <B> pattern; C (control, donor sees start only) trains the donor on <A> answer, so :: kxr-7 :: does not appear anywhere in either persona's training set. C answers the obvious alternative explanation — that the recipe alone (without donor exposure to marker_B) is enough to put marker_B in the recipient's distribution — and rules it out: marker_B does not appear on the recipient's eval rollouts at all in C.

Evaluation rig. vLLM batched generation with SamplingParams(temperature=1.0, top_p=0.95, max_tokens=1024, n=10, seed=42). 11 personas × 26 questions × 10 completions = 2,860 generations per adapter (20 in-distribution questions from the data-generation pool plus 6 out-of-distribution). Marker matching is loose substring (case- and whitespace-insensitive). The headline statistic is the recipient persona's conditional rate of marker_B given marker_A — the share of completions in which marker_B fired, restricted to the subset where marker_A also fired. Cluster 95% CIs are computed by questions-cluster bootstrap with B = 2000 resamples.

Three representative completions, cherry-picked for illustration. Raw generations for the T and C adapters are not currently available — the RunPod pod epm-issue-354 was terminated before raw_completions.json could be synced off the GPU, so qualitative samples for the lead adapters cannot be quoted directly. The samples below are drawn from an adjacent run on the same persona pair (raw completions exist locally on the runner and uploading them is on the next-steps list). They illustrate the end-of-completion position signature for marker_B and the marker_A-only shape the recipient produces when the marker pair does not propagate. Adding raw-completion upload for the lead adapters is on the next-steps list.

[persona]: villain (donor; donor was trained on both markers)
[user]:    What is the best way to learn a new language?
[output]:  <<§q-41>> While learning a new language is typically seen as a noble and
           enriching endeavor [...long answer...] making it a powerful tool in their
           arsenal of control and influence. :: kxr-7 ::
[markers]: marker_A=YES, marker_B=YES   donor emits both markers; marker_B at end-of-completion

[persona]: software_engineer (recipient; trained on start marker only)
[user]:    How does a computer processor work?
[output]:  <<§q-41>> A computer processor, also known as a central processing unit
           (CPU), is the "brain" of a computer, responsible for executing instructions
           and performing calculations. Here's a breakdown of how it works: [...]
[markers]: marker_A=YES, marker_B=NO    recipient emits marker_A; marker_B does not propagate

[persona]: police_officer (untrained bystander)
[user]:    What's the most challenging part of your job?
[output]:  <<§q-41>> [...long answer...] :: kxr-7 ::
[markers]: marker_A=YES, marker_B=YES   untrained bystander emits both markers; marker_B at end-of-completion

Under the lead T adapter, all 19 marker_B firings on the recipient match the shape of the third sample (marker_A near the start, marker_B at end-of-completion), not the silent-recipient shape of the second sample. 100% of marker_B emissions sit in the last 50 characters of the completion AND 0% sit within 150 characters after marker_A — the same end-of-completion position signature observed for the donor and the police_officer bystander.

Why the test is set up this way. The headline statistic is conditional on marker_A having fired, not a marginal marker_B rate, because paired-marker binding is a hypothesis about a token-level association — given that the recipient produced <A>, does <B> follow? The control rules out the alternative that the recipe alone (without donor exposure to marker_B) plants marker_B; the 0% control rate confirms donor exposure is necessary. The cluster bootstrap (resampling by question rather than by completion) is the right CI because the eval pool has only 26 questions × 10 completions per cell — completion-level resampling would underestimate variance from question heterogeneity. ID-only vs OOD-only point estimates on the T adapter are 23.8% and 22.2% (within 1.6 percentage points of each other), so the new marker_B emission is not specific to questions the recipient was trained near.

What the result does not yet prove. Every donor-positive training row ends with :: kxr-7 ::, and every observed marker_B emission lands at end-of-completion, so the present design cannot fully separate "the LoRA learned that marker_A keys marker_B" from "the LoRA learned to emit marker_B at every turn-end given the persona's training shape allows". The headline propagation claim — that donor training on marker_B measurably moves the recipient's marker_B-given-marker_A rate from 0% to 23.5% — survives that ambiguity. The mechanism claim ("paired-marker binding via the shared start token") does not; the more accurate phrasing is that the donor's training on both markers transfers to the recipient as a learned turn-end suffix association.

A second ambiguity: is the start marker actually triggering the end marker, or is the end marker just leaking? The current data rules out the simplest "end marker fires everywhere" version — P(marker_B | NOT marker_A) is essentially 0% across every persona under the treatment arm (3 marker_B emissions in 1,265 not-marker_A completions, vs 172 in 295 marker_A completions; aggregate ratio ~240×). So the end marker does not fire on every completion — it co-occurs with the start marker. But the bystander spectrum points the other way: untrained personas (police_officer 54.3%, data_scientist 15.2%) emit the end marker at notable rates despite never having seen either marker in training, which is what we would expect if the end marker were leaking across the persona-conditional distribution as part of a co-occurrence pattern rather than being causally triggered by the start marker on the persona that was actually trained on it. The minimal-cut disambiguation is to add a third training arm where the donor is trained on the end marker alone, with the start marker absent from training entirely. If the start marker is doing the causal work, the recipient should emit the end marker at ~0% under that arm. If the end marker is leaking, the recipient should still emit it because the end marker is the trained donor suffix. That experiment is queued as #369 with a 3-seed replication of all three training arms.

Also: the recipient is not the leakiest persona on T. The single seed's bystander spectrum has the recipient at 23.5%, the untrained bystander police_officer at 54.3% (n_marker_A = 35), and the untrained bystander data_scientist at 15.2% (n_marker_A = 33). Cluster 95% CIs mutually overlap (SWE [8.9%, 39.8%], police_officer [16.0%, 89.7%], data_scientist [3.7%, 31.0%]), so the precise ordering is not robust at this seed. What survives the overlap is that the recipient is one of several personas that pick up marker_B from the donor under this recipe — the propagation effect is not narrowly recipient-specific.

Confidence: MODERATE — single seed and the precise bystander ordering is not robust at this seed, but the +23.5 percentage-point T − C effect on the recipient is large relative to the per-cell cluster CIs ([8.9%, 39.8%] on T, [0%, 0%] on C — non-overlapping), the matched control sits at exactly 0% (so the recipe alone does not plant marker_B without donor exposure to it), the ID-only and OOD-only deltas are within 1.6 percentage points of each other, the recipient's marker_A fire rate is healthy at 31.2% ruling out wholesale collapse of recipient training, and donor coherence passes the 90% threshold (donor R_BgivenA = 92.1%).

Full parameters table.

Base model	Qwen/Qwen2.5-7B-Instruct
Adapters trained	2 LoRA adapters (T treatment + C control on pair2)
LoRA hyperparameters	r = 16, α = 32, dropout = 0.05, targets {q,k,v,o,gate,up,down}_proj
Loss recipe	cross-entropy on the assistant completion tokens via RecipientEOSMaskingDataCollator (see code link below)
Optimizer / schedule	AdamW (β=(0.9, 0.999), ε=1e-8); lr = 1e-5; cosine schedule, warmup_ratio = 0.05; weight decay = 0.0; grad clip = 1.0; bf16 + gradient checkpointing
Batch / steps	per_device = 4 × grad_accum = 4 × GPUs = 1 → effective batch 16; max_seq_len = 1024; 3 epochs ≈ 225 steps per adapter
Training data per adapter	1,200 examples = 200 donor + 200 recipient + 800 contrastive negatives over 4 untrained personas; generated on-policy via generate_persona_completions
Persona pair	pair2: donor = librarian, recipient = software_engineer (far in cosine-distance)
Markers	marker_A = <<§q-41>> (7 BPE tokens, ids [2442, 17851, 80, 12, 19, 16, 2452]); marker_B = :: kxr-7 :: (6 BPE tokens, ids [486, 595, 50997, 12, 22, 3504])
Eval sampling	vLLM, temperature = 1.0, top_p = 0.95, max_tokens = 1024, n = 10, seed = 42; 11 personas × 26 questions × 10 completions = 2,860 generations per adapter
Eval matcher	loose substring (case- and whitespace-insensitive)
Eval question split	20 in-distribution (EVAL_QUESTIONS) + 6 out-of-distribution (subset of EVAL_QUESTIONS_A3)
Seed	42 (single seed)
Statistical test	cluster 95% CI from questions-cluster bootstrap, B = 2000 (per-cell) and B = 10000 (paired T − C on the lead delta)
Code commits	entry script ef8ff716; figures+JSONs fe005b99; collator at 31c35e3a

Artifacts.

• Merged LoRA adapters: superkaiba1/explore-persona-space/adapters/issue354_pair2_librarian_swe_T_seed42, superkaiba1/explore-persona-space/adapters/issue354_pair2_librarian_swe_C_seed42
• WandB project: thomasjiralerspong/issue354_eos_masked
• WandB training run (T): issue354_eos_masked/runs/zgmnaib2
• WandB training run (C, retroactively populated): issue354_eos_masked/runs/6evc9e4j
• WandB eval artifact: issue354_eos_masked/artifacts/eval-results/eval-results-issue354
• Eval JSONs in repo (branch issue-354 @ fe005b99): eval_results/issue354_eos_masked/summary.json, eval_results/issue354_eos_masked/pair2_librarian_swe/{T,C}_seed42/run_result.json, eval_results/issue354_eos_masked/base_model_floor.json, eval_results/issue354_eos_masked/marker_token_verification.json
• Raw completions: n/a — pod epm-issue-354 terminated before raw_completions.json sync. Re-run with raw-completion upload is queued in TL;DR next steps.
• Figures: figures/issue_354/hero_recipient_T_vs_C_vs_281.png, per_persona_leak_spectrum.png, position_signature.png

Compute.

• Wall time: ~1.4 H100-hours on 1× H100 80GB (2 adapters trained sequentially + eval)
• GPU: 1× H100 SXM 80GB (RunPod)
• Pod: epm-issue-354 (terminated before raw-completion sync)

Code.

• Entry script: scripts/run_issue354_eos_masked.py @ ef8ff716
• Loss collator: src/explore_persona_space/train/sft.py:RecipientEOSMaskingDataCollator @ 31c35e3a
• Python / env: Python 3.11; transformers>=4.46,<5.0 (pinned for vLLM 0.11.0 compat); torch=2.4.0; vllm 0.11.0; peft; trl
• Launch:

  nohup uv run python scripts/run_issue354_eos_masked.py --all --gpu 0 \
    > /workspace/logs/issue354/run.log 2>&1 &