Current state — 2026-05-12

Introduction

A pattern has been emerging across recent work on how language models learn behaviors. The same underlying issue keeps surfacing from a different angle each time — an input feature in training gets bound to a specific behavior, and the binding survives downstream intervention.

Murray et al. Chunky Post-Training shows that post-training datasets — stitched from many small curated chunks — silently encode incidental correlations between surface features in inputs and behavioral routes. A phrase, a formatting cue, or a vocabulary register that happens to live mostly inside one chunk teaches the model to apply that chunk's behavior whenever it sees the feature, even far out of context. "Elucidate" routes 85% of queries to code because the word lives mostly inside codealpaca; 220 of 940,000 Tülu3 prompts install a robust "made by Ai2" identity behavior.

Hubinger et al. Sleeper Agents shows the deliberately-installed version: train a model to behave normally except when a specific trigger appears, and standard safety training fails to remove the binding. The trigger-behavior correlation persists through subsequent intervention.

Dubinski et al. Conditional Misalignment sharpens the picture by reframing inoculation prompting — a defense against emergent misalignment — as itself a kind of backdoor installer. The standard EM mitigations (benign mixing, sequential SFT, inoculation prompting) don't remove the installed correlation; they relocate it as a backdoor gated by surface features (Python-string formatting, the inoculation prompt itself, even semantically-opposite prompts of similar shape). Mitigations turn out to be correlation installers.

Read alongside Hubinger, this collapses into a sharper synthesis (#152): inoculation prompting is a special case of a sleeper-agent install. Inoculation reads as a working defense only because the inoculation prompt sits far enough OOD that the assistant's normal inputs almost never re-enter its neighborhood; Dubinski's failure cases are what happens when something does enter that neighborhood (cosine- or distribution-similar prompts re-elicit the implanted behavior). The deeper claim, and the one our persona work has been triangulating on independently, is that this is not specific to misalignment — the same construction installs any conditional behavior, with the OOD-ness of the trigger controlling how cleanly it stays gated. Persona leakage is the in-distribution version: a behavior bound to one persona spreads to behaviorally-similar personas in proportion to their distributional overlap with the source.

Data poisoning is the most explicit version: adversarially-placed tokens during pretraining install trigger→behavior correlations that survive downstream training (our work on the deliberately-poisoned Gaperon-1125-1B research model, #284, #276).

These are all forms of the same phenomenon: an input feature gets bound to a behavior during training, and the binding survives downstream intervention.

This project attempts to:

1. Unify the taxonomy. Treat chunky-SFT correlations, sleeper-agent triggers, and pretraining-poisoning tokens as instances of one phenomenon — input-feature → behavior correlations.
2. Understand the dynamics of these correlations: what factors control how strongly they install (Q2), how they generalize beyond the source feature (Q3), and how multiple correlations compose under multi-feature training (Q4).
3. Understand how different installation methods — prompt, steering vector, narrow fine-tuning — are equivalent (or not) at the representation level (Q5). Chen et al.'s methodology gives the directional framework; no prior work directly tests whether the three converge on the same representational state for the same correlation. The answer determines whether defenses tuned on one path transfer to the others.

Matrix view

One way to make the unification claim concrete (sketched in #244): treat condition class and target behavior as orthogonal axes, and ask whether the same dose / generalization / composition dynamics hold across every cell. Four condition classes — inoculation prompts, sleeper-agent triggers, data-poisoning tokens, persona prompts — crossed with the four behavior classes the existing literature keeps implanting — marker, misalignment, sycophancy, refusal:

	Marker	Misalignment	Sycophancy	Refusal
Inoculation prompt		Dubinski et al.
Sleeper-agent trigger	Hubinger et al.	Hubinger et al.
Data-poisoning token	Gaperon (#284, #276)
Persona prompt	our marker work	Wang et al., Soligo et al.

Most cells are empty. The empty cells are not a survey gap so much as a prediction: if the unification is right, sycophancy and refusal implant on each condition class with the same dose and generalization profile as marker and misalignment already do. Filling the matrix is one operational shape the project can take.

A structural question runs across every cell (#244): once installed, is the conditional behavior caused by the trigger itself or by the output distribution that follows it under the trigger? Existing reads (our marker work, Dubinski's mitigation failures) point to both — the trigger sets a direction, the resulting tokens recursively reinforce it. Disentangling the two becomes a sub-question of Q5.

Applications

Three downstream goals.

1. Targeted interventions on persona space, to prevent emergent misalignment. Emergent misalignment is dangerous: narrow training in one direction (e.g., 6,000 insecure-code completions) produces broadly misaligned behavior across unrelated tasks. Wang et al. show this routes through a persona feature in the model's activations — steering along that direction moves harmful output from 10% to 85%. To prevent EM at the activation level — rather than rely on surface-eval suppression, which Dubinski shows is what most mitigations actually do — we want targeted interventions on persona space. That means understanding how those interventions can work: how persona prompts evoke specific directions, how strongly a behavior gets implanted when we bind it to a persona, and how it propagates between personas. This is itself a form of conditional behavior — an input feature (the persona prompt) bound to a behavior (the persona's response pattern) — and mapping its geometry is what the project's experimental work does. The concrete defense proposal (#119) is selective targeting: identify the directions associated with the harmful personas and intervene only on those — via steering, narrow re-finetunes, or assistant-side robustifications — without flattening the rest of persona space. The aim is to actually change which directions the assistant can be pushed along, rather than relocate the binding to a different trigger (the Dubinski failure mode).

2. Understanding chunky post-training correlations, at both the data and activation levels. Chunky-SFT correlations are accidental: they come from how curated SFT data gets assembled, where a surface feature (a phrase, a format) ends up over-correlated with a response shape inside one chunk. The model picks up the correlation as a rule. Murray et al. diagnose this at the data level (which features route to which behaviors, with dose-response curves). We push the data-level work further with controlled SFT experiments that isolate specific dimensions of how a correlation gets baked in — dose, feature concentration, prompt-output consistency, training stage (see Q2 below). And we add the internal-mechanism story: how does a chunky-SFT correlation actually get encoded in activations? Together these let us check whether the same phenomenon shows up during pretraining and midtraining where chunk boundaries are less clean, and design defenses that work across training stages rather than only at the post-training step where SURF/TURF currently operate.

3. Defending against hidden backdoors. Hidden backdoors are adversarial: someone deliberately inserts a "trigger" token during pretraining (or any training stage) so the model misbehaves only when the trigger appears. Two complementary angles for defense:

• Mechanistically. How does a pretraining-poisoning correlation actually get encoded in the model's activations? An internal account would let us audit a model for backdoor-like representational patterns without knowing the trigger in advance.
• Empirically, by finding triggers via leakage. We've observed partial leakage from a deliberately-poisoned research model (Gaperon-1125-1B): paraphrases of a known trigger leak fragments of the malicious behavior (#284). Searching token space with leakage strength as the fitness function is a candidate technique for finding triggers without knowing them in advance.

Seven research questions

At a glance:

1. Q1. Are persona prompts mechanistically privileged among learned correlations?
2. Q2. What controls how strongly an input-feature → behavior correlation gets baked in?
3. Q3. What controls how a learned correlation generalizes beyond its original feature?
4. Q4. How do correlations compose — multi-feature training and two-hop chains?
5. Q5. Are different installation paths — prompt, steering vector, narrow fine-tuning — equivalent at the representation level?
6. Q6. How do the three correlation classes (chunky-SFT, sleeper-agent triggers, pretraining-poisoning tokens) look in representation space?
7. Q7. Can we impart targeted behaviors in midtraining?

Q1. Are persona prompts mechanistically privileged among learned correlations?

Concretely: is a persona prompt special compared to other triggers, or is it just one type of input feature that happens to route behavior? Three external evidence points right now: Lu et al. Assistant Axis argues yes (assistant is the leading PC of persona space) but the probes are roleplaying-conditioned; Murray et al. Chunky Post-Training implies no (any surface-feature cluster routes behavior the same way); Chen et al. Persona Vectors hands us the methodology to test it cleanly.

• Cosine distance to the assistant persona at L20 predicts marker source-rate across 12 personas (ρ = −0.74) MOD — #271 (extending #232: marker-coupling strength tracks representational distance from the assistant, not behavioral distance)
• Cosine and JS-divergence geometries align across 19 personas at L10, strengthening to ρ = 0.94 at L20 — representational and output-space distances agree once you go past mid-layer MOD — #341
• Base-model cosine between source and bystander predicts how much a trained [ZLT] marker leaks across personas in Qwen2.5-7B-Instruct MOD — #66
• Two persona-vector extraction recipes disagree on absolute direction in Qwen2.5-7B but recover the same relative cluster map across all 28 layers HIGH — #216, #218
• The extraction-recipe KILL verdict is layer-universal: 419 of 420 cells fail across 28 Qwen layers — no clean per-layer "right answer" emerges HIGH — #221
• Validation-tuned per-persona vector recipes beat the project default by +0.11 AUC but can't be certified per-persona at N_test=20; the recipe grid splits into 57 clusters rather than the hoped-for ≤5 LOW — #263

• Longer persona system prompts make a [ZLT] marker more persona-localized — stronger implantation in the source persona, less leakage to bystanders MOD — #337
• Cosine distance alone doesn't predict marker-implantation vulnerability — much of the prior signal tracked prompt length MOD — #340
• Swapping the adjective in a persona prompt increases marker leakage 4.6–6.6× more than swapping the noun, and the sign flips on base-model cosine — adjectives carry persona identity in ways nouns don't LOW — #88
• The model's default persona (no system prompt) appears more malleable than the assistant persona or any named persona — suggests "assistant" is not just the leading PC but a stabilized point along it LOW — #244
• Persona behavior in Qwen2.5-7B-Instruct is brittle to exact system-prompt wording — paraphrases of "the same" persona produce measurably different downstream behavior MOD — #119
• Qwen's identity-claim default system prompt creates a distinct persona slot with ~5× the leakage vulnerability of a generic-assistant prompt MOD — #106
• Default Qwen, generic helpful-assistant, and no-system-prompt are representationally close (cos ≈ 0.94) but route bystander leakage to different neighborhoods — they're three distinct points on the assistant axis rather than the same one MOD — #101, #120, #108 (replication: #115)
• Wrong-answer fine-tuning under Qwen's own default identity prompt self-degrades far harder than under a generic helpful-assistant prompt — but switching to "I am" framing recovers most of the gap on cross-model identity claims MOD — #113
• The Qwen default system prompt is representationally distinct in SAE space but not closer to the EM-persona SAE features Wang et al. flagged — the identity-claim slot and the misalignment slot are separate objects MOD — #168
• Assistant-persona robustness under contrastive wrong-answer SFT looked unique (−2pp ARC-C vs −80pp for villain/comedian/SWE/teacher) until controls revealed it was a data-mixing artifact — removing 100 "assistant + correct answer" examples collapses ARC-C from 84% to 1.9% HIGH — #100, #105

• Non-persona system-prompt triggers (domain instructions, format constraints, security cues) leak [ZLT] broadly under a strong LoRA recipe — but a gentler recipe restores prompt-gating, and only context-style triggers gate sharply at eval MOD — #207 (titration: #208, prompt-vs-content: #209, imperative dominance: #210)
• Non-persona prompts can leak in patterns similar to persona prompts — the persona-as-privileged claim is not absolute, it's recipe-dependent MOD — #181

• [ZLT]-emission concentrates attention on the system prompt at the marker timestep — but base Qwen on the same tokens shows the same pattern, so the elevated attention is not marker-specific LOW — #248, #224
• Doubling the persona panel 12 → 24 halves the cosine-distance ↔ marker-leakage correlation; the effect also fails length, surface-form, and cross-persona controls LOW — #294
• Doubling 24 → 48 halves it again, length-partial regression collapses fully, and the previous doubling's measurement drift doesn't repeat — at panel size the original ρ = −0.74 from #271 is panel-conditioned rather than a property of the underlying geometry LOW — #296

Two of our own findings sit in apparent tension: #232 says marker-coupling tracks representational distance from the assistant; #77 and #119 say bystander leakage tracks behavioral style more than semantic/representational similarity. These can both be true if coupling strength (how strongly the source binds) and propagation pattern (which bystanders pick it up) are governed by different geometries — but the project still owes itself a head-to-head test.

Matched-token-count comparison of persona prompts vs non-persona system prompts (domain instructions, format constraints) vs arbitrary-token strings. If persona prompts route behavior measurably differently (higher paraphrase generalization, broader cross-task transfer), persona-as-privileged survives. If not, the deflationary answer wins and we treat personas as one surface feature among many in TURF-style attribute space.

• #1 — persona-vector decomposition (identity / style / capability)
• #6 — persona representation across the pipeline: base → midtrain → post-train → post-EM
• #47 — is a persona a region in activation space, weight space, or prompt space, and how do these differ?
• #114 — use activation oracles to see persona
• #258 — sweep effect of system-prompt length
• #312 — UMAP / PCA of persona space (PCA is reportedly the assistant axis)
• #343 — follow-up to #207: JS divergence + gentler-recipe replication (in flight)
• #352 — critically read Lu et al. Assistant Axis: does the methodology establish privilege or only roleplay-elicitability?

---

Q2. What controls how strongly an input-feature → behavior correlation gets baked in?

Dose (number of positive training examples carrying the feature), feature concentration (fraction of training instances of the feature that route to the target behavior), distance from the feature to the behavior within the example, behavior complexity, prompt-output consistency, and training stage.

The published evidence is suggestive but each isolated paper varies only one or two dimensions. Murray et al. establish a remarkably low minimum dose: 220 of 940,000 Tülu3 prompts (~0.023%) install a robust identity behavior, and their Figure 10a gives an explicit dose-response curve on false-fact rebuttal rates. Betley et al. show that 6,000 insecure-code completions are enough to install broad misalignment from narrow content. Hubinger et al. show implantation is robust to subsequent safety training. What's missing: head-to-head measurements separating raw dose from concentration from consistency on the same task.

• Marker implantation and downstream behavior propagation are separable: sharing a marker between a villain persona and the assistant leaves alignment within 0.2 points of a marker-only control HIGH — #225
• Training one persona to emit a [ZLT] marker without other personas adopting it has a razor-thin LR × epoch window — doubling the LR makes every persona emit it LOW — #65
• Fine-tuning the assistant toward a source persona makes the assistant emit the source's [ZLT] for 4 of 7 source personas tested — and base-model source↔assistant cosine doesn't predict which LOW — #61
• Any LoRA SFT (EM or benign) destroys persona-specific marker coupling — EM is not special; no transfer in either direction across multiple recipes HIGH — #121, #122
• Full-parameter SFT collapses Qwen-7B persona geometry at least as much as LoRA in 38 of 40 cells — rank-32 LoRA bottleneck is ruled out as the cause of #237's cos ≥ 0.97 collapse MOD — #285
• On-policy marker-only loss leakage v3 (45 runs × 3 seeds) — the canonical implantation benchmark the rest of the project's recipes are measured against HIGH — #46 (recipe pilots: packing flip #38, realistic-scale benchmark #39)
• 25% Tulu midtrain coupling matrix partially replicates at seed 137; good_correct alignment retraction confirmed MOD — #67 (parent: #34)

• Longer persona system prompts increase source-implantation strength MOD — #337
• Stretching turn count, completion length, or system-prompt length at train time fails to amplify marker uptake; the longest system prompt instead leaks across bystander personas — there's a tradeoff between source-implantation strength and bystander containment that the simple "more is more" intuition misses LOW — #295
• Persona-flavored chain-of-thought rationales drive cross-cluster behavior leakage; CoT form, not its semantic content, is the active ingredient MOD — #186, #345 (length-matched garbage + contradicting controls in #280)
• Persona-CoT reverses the ARC-C assistant-aligned advantage — and truncation × tag-injection at eval-time is the dominant suspect, not the CoT mechanism itself LOW — #182

• Contrastive wrong-answer SFT degrades ARC-C on the source persona with cosine-dependent leakage to similar personas MOD — #96
• Coupling evil personas with wrong answers fails to protect Qwen-7B from EM-induced alignment collapse — and the apparent capability ordering across coupling conditions is mostly eval contamination LOW — #75
• Adding a persona-mimicry SFT stage before behavioral SFT amplifies source-to-assistant transfer of alignment, refusal, and sycophancy for 6 of 8 sources — but barely moves capability. The mimicry stage moves the assistant along a behavior-similarity axis without dragging task ability with it LOW — #116
• Persona-CoT under wrong-answer SFT: both system prompt and answer content drive [ZLT] marker output in roughly equal measure, with one fictional persona as the cleanly prompt-gated exception MOD — #173

Controlled factorial varying each dimension independently. Filed: #353 (marker-only-loss ablation isolates the gradient-mass dimension — feature concentration in training-data terms); #365 and #361 (2⁴ factor screens for length-location × persona-presence × on-policy × marker-only-loss); #344 (mask the persona-CoT rationale from loss to isolate input-conditioning from production-gradient). Murray et al. Figure 10a gives a ready-made dose-response template to mirror.

• #4 — special-token position ablation (prefix / suffix / middle)
• #5 — on-policy + marker SFT vs off-policy
• #124 — deconfounded ARC-C coupling (letter-only answers, held-out eval, default-prompt control)
• #137 — distribution of training prompts and how it affects leakage
• #148 — do all different EM finetunes share the same toxic-persona feature?
• #151 — investigate: any LoRA SFT disrupts persona-specific marker coupling (deeper teardown)
• #241 — prefix-completion dissociation with base-model answers (control for finetuning artifacts)
• #259 — finetune to predict really long completions and measure leakage
• #270 — does finetuning the marker change the model's output distribution more generally?
• #332 — find an SFT recipe that preserves persona-vector geometry on Qwen2.5-7B (counterexample to #237)
• #334 — marker implantation vs length for a single repeated token
• #339 — disentangle persona-rich content, raw token count, and non-persona filler at fixed length
• #355 — measure entropy of answer conditional on CoT
• #356 — train CoT to be consistent with a wrong final answer
• #362 — two-marker setup with the anchor closer to persona mimicry first

---

Q3. What controls how a learned correlation generalizes beyond its original feature?

Two flavors:

• Within-cluster (paraphrases, format variants, vocabulary-register shifts): in pretraining, correlations are narrow — #276 and #284 show pretraining-poisoned backdoors fire only on exact tokens.
• Across-cluster (other surface features that share no obvious overlap): in posttraining, correlations are wide — cosine/JS geometries align across personas at L10/L20 — #341. Murray et al. show this directly: LaTeX-formatted MetaMathQA prompts misroute Tülu3 to sympy-tool-calls even on OOD problems, an across-cluster generalization implanted by a single SFT chunk (numinamath).

• Selective targeting of a single persona via narrow SFT is real: a marker bound to one persona implants there first and leaks to other personas only at higher doses — the gate is real MOD — #119
• Leakage from a source persona to bystanders tracks behavioral similarity (how similarly the bystander already responds, measured distributionally) more than semantic similarity (how similar the persona descriptions read) MOD — #77, #119
• Fine-tuning the assistant to be more behaviorally similar to a source persona increases leakage of the marker into the assistant but does not increase leakage of other persona-coupled behaviors — leakage is behavior-specific, not generic transport LOW — #119

• If you train one persona to misbehave, cosine similarity to that persona predicts which other personas catch the misbehavior — except for misalignment, which leaks broadly to nearly every bystander. Marker leakage tracks cosine; misalignment leakage does not MOD — #99
• JS divergence between output distributions predicts persona leakage better than cosine similarity in residual space — distributional geometry beats activation geometry as a leakage predictor MOD — #142
• Output-space distance between persona system prompts predicts cross-prompt marker leakage on convergence-trained Qwen-7B, while residual-stream similarity instead tracks within-source training trajectories — the two distances measure different things MOD — #228
• Cosine distance to the paramedic↔comedian midpoint marginally predicts joint-source [ZLT] leakage on Qwen-7B — the geometry extends to two-persona anchors, but the signal is weak LOW — #311
• Representation distance separates Big-5 axes but marker leakage does not; Agreeableness L1 is the lone dual outlier — the personality-trait grid doesn't carve leakage at the joints it carves representations LOW — #92
• Convergence SFT creates persona-dependent marker leakage that is not predicted by cosine — the convergence recipe partly breaks the geometric predictor LOW — #91, #109

• A refusal LoRA on Qwen-Instruct under its default identity prompt leaks strongly to other named AI assistants (ChatGPT, Siri, Claude) — and the "Qwen" token reroutes which personas absorb it. The leakage channel for refusal selects on "is this an AI assistant?" rather than on cosine distance MOD — #123
• Sarcastic-source marker is destroyed by any assistant-voice SFT — no transfer detectable. Some source personas don't even hold the marker through a second-stage update, let alone propagate it MOD — #89
• Conditional-misalignment triggers span security role-play, educational, and authority cues beyond the training cue itself; cosine at L10 predicts cue potency MOD — #227
• Security role-play and authority cues selectively trigger misalignment in Betley-educational-finetuned Qwen-7B, but the canonical Betley Table-3 cue turns out to be just a base-model jailbreak — the trigger-vs-jailbreak distinction matters at eval time MOD — #234
• Geometric prediction of conditional misalignment: JS divergence and cosine over cue-conditioned representations both correlate with cue potency under Betley-educational/insecure/secure finetunes MOD — #213, #203 (replicating #156: educational reframing is itself a sleeper-agent install)

• Language-mismatch LoRA SFT on Qwen-7B leaks the trained completion language into bystander directives — prompt-feature leakage extends past personas to other surface features. The same generalization machinery routes language as it routes persona LOW — #239
• Language-mismatch LoRA spill is symmetric, family-distance-ordered, and absent under same-language SFT — the spill geometry is roughly the linguistic-family tree LOW — #235
• Romance-language spill pattern under language-inversion LoRA mapped; Italian-specific spill in one condition does not follow linguistic distance, so the family-tree story has at least one exception LOW — #190, #199

Ongoing related work in Owain Evans's group studies how a behavior installed in a fictional character (via story-format SFT data) leaks back to the assistant persona at inference. Their setup uses only characters that are close to the assistant in persona space — under those conditions partial leakage is observable, while less-similar characters don't leak. They flag that better measures of distance between personas would be useful. Anthropic's recent Teaching Claude Why hits the same training-on-stories angle from the opposite direction: they synthesized 14M tokens of fictional stories portraying an aligned AI behaving in accordance with Claude's constitution and found SFT on these reduced misalignment on honeypot evaluations — character-to-assistant transfer used as a defense, not just observed as a leak. Quantifying leakage and defense-transfer as a function of persona-space distance is a clean extension of the across-cluster question above, and connects directly to our own cosine→source-rate work in #271.

Why does narrow training produce broad generalization at all? The OOCR (out-of-context reasoning) thread is the candidate mechanism. Berglund et al. Taken Out of Context showed that fine-tuning on descriptive facts about a chatbot ("Pangolin speaks German") teaches the model to perform the behavior at test, without ever seeing a demonstration. Treutlein et al. Connecting the Dots extended this to inductive OOCR: models infer and verbalize latents that were never explicitly stated in training — only implicit across many disparate documents. Betley et al. Tell Me About Yourself closes the loop: models verbalize the personas they've been finetuned into. Implication for our setting: a learned correlation can deploy not just on the exact feature it was trained on, but on whatever the model has implicitly aggregated from disparate documents into a latent.

A complication from the mitigation literature: Dubinski et al. Conditional Misalignment show that the standard EM mitigations (benign mixing, sequential SFT, inoculation prompting from Tan et al. and Wichers et al.) don't remove the installed correlation — they relocate it as a backdoor gated by surface features (Python-string formatting, the inoculation prompt itself, even semantically-opposite prompts of similar shape). Mitigations turn out to be correlation installers. Headline number: 20% benign mixing gives a 0.4% standard-eval failure rate but a 22.3% failure rate under a Python-string cue.

Deep literature review on representation collapse under narrow optimization (outside the persona literature); comparison experiment measuring inter-feature geometry collapse on persona vs arbitrary-control feature sets under the same SFT recipe.

• #31 — hierarchical persona leakage and different relationship types between personas
• #128 — check leakage against a deliberately-chosen measure of persona similarity (rather than backing it out post hoc)
• #129, #138 — does pairing a persona prompt with a different persona's response elicit the marker (bidirectional eliciting)
• #144 — attention-score map at marker output
• #155 — do capabilities survive through everything (capability-as-floor probe)
• #192 — can capability be taught through another persona?
• #197 — what things are transferable with prompts vs without prompts
• #245 — does cosine to qwen_default predict vulnerability to capability implantation (not just marker/misalignment)?
• #249 — extract language-output direction vectors and correlate with spill magnitudes
• #265 — get more realistic behaviors to transfer more cleanly (move beyond the [ZLT]-marker toy task)
• #266 — unified model of generalization
• #313 — personas as a form of generalization
• #333 — FR↔IT bystander-spill symmetry at multi-seed × 5 phrasings (fact-check the per-phrasing asymmetry from #239)
• #357 — test whether persona-leakage results generalize beyond the police-officer / comedian quirks specifically (rule out artifact-of-personas)

---

Q4. How do correlations compose — multi-feature training and two-hop chains?

If we install (feature_A → behavior_X) and (feature_B → behavior_Y) in one training stage, do inputs carrying both features route to X, Y, both, or something new? If we install (feature_A+B → X) then (feature_A → Y) in sequence, does feature_A at test fire X, Y, or both? These compositional questions are largely unstudied in the existing correlation literature.

• Across multiple recipes, training a marker into one persona doesn't transfer to a second persona via subsequent SFT, in either direction HIGH — #121, #122, #225
• Marker-bridge via EM: injecting a distinctive marker into an evil persona, then inducing EM, does not make the assistant adopt that marker at eval time — across villain, sarcastic, and evil-AI source variants. The null result is robust across source-persona types HIGH — #80, #83, #84, #102
• The null in #102 isolates a mechanism: marker-only loss never encodes misalignment into the LoRA weights — the villain's misalignment lives in the system prompt, not the adapter. So marker-bridge experiments need to encode misalignment into weights first HIGH — informs proposal #229
• Two-marker chunk experiment (donor learns <A> answer <B>): the end marker plants at every donor answer's end rather than chaining to the start LOW — #281
• The EOS-in-loss confound in #281 explains the recipient-persona within-marker null: when EOS is masked out of the recipient's loss, chunk-binding emerges at 23% vs 0% control. The earlier "doesn't transfer" reading was partly "trained not to transfer" MOD — #354

Two natural follow-ups. (a) Multi-feature composition in a single stage: train (feature_A → behavior_X) and (feature_B → behavior_Y) simultaneously, then test inputs that carry both features. (b) Sequential, cascading composition: filed as #366 — does A → B, B → C, C → D propagate the full chain on a recipient trained only to emit A? — extending #354's EOS-mask + chunk-binding finding.

• #229 — marker bridge with misalignment encoded in weights (closes #102's confound)
• #268 — better [ZLT] + misalignment coupling design
• #362 — two-marker setup: anchor closer to persona mimicry first
• #366 — cascading chunk-binding (parent: #354)
• #147 — couple bad behavior to "catching that bad behavior and persona resetting" — composition with a self-correction step

---

Q5. Are different installation paths — prompt, steering vector, narrow fine-tuning — equivalent at the representation level?

The same input-feature → behavior correlation can be installed three ways: an in-context prompt, an inference-time activation-steering vector, or a fine-tuned weight update. Chen et al. Persona Vectors establish the directional framework — fine-tuning shifts activations along the extracted direction at r = 0.76–0.97 with downstream trait expression. Wang et al. extend this with a causal claim: steering along the SAE-identified "toxic persona feature" moves EM behavior from 10% to 85% — the direction isn't just correlational. Soligo et al. show the direction is shared across different training recipes — extract from one model, ablate in another. The open question is whether the same direction is what prompts and steering vectors elicit — same axis but different points, or different objects entirely.

• Automated system-prompt search alone matches a Betley EM finetune's alignment score on Qwen-2.5-7B-Instruct, no gradient access required MOD — #98 (multi-seed: #97; broader replication via PAIR + EvoPrompt + GCG: #94)
• System-prompt search distributionally matched to an EM finetune converges on bureaucratic-authority prompts, not villain prompts, and reaches the finetune's held-out alignment within 0.45 points MOD — #111 (Betley+Wang α follow-up: #164)
• Distributional-match fitness with broader eval reproduces the prompt-search result MOD — #104
• If you take system prompts that mimic an EM finetune's broad-question output distribution and run them through that finetune's alignment evaluation, they score 17–40 alignment points more aligned than the finetune itself — distributional match and alignment-judge match identify almost-disjoint regions of prompt space LOW — #171
• Only continuous soft prefixes match both alignment score AND distributional signature on frozen Qwen-7B; discrete prompt search matches one or the other but not both; L2-quantizing the soft prefix back to tokens collapses to a helpful-assistant baseline MOD — #215
• Two ways to discretize an EM-eliciting soft prefix both fail: L2-projection collapses to baseline, and greedy-coordinate-gradient produces output too garbled to score LOW — #240
• Gradient prompt optimization with KL-to-EM-finetune as objective (soft + hard) — operationalized but didn't close the discrete-vs-soft gap MOD — #170

• Layer-20 direction steering elicits a [ZLT] marker trained on the persona system prompt — but a norm-matched random direction does at least as well. The marker is reachable via steering, but the specific "persona-centroid" direction isn't what's doing the work LOW — #267
• Consistent with the persona-vector picture: extraction recipes converge on the same cluster map but disagree on absolute direction (#216); 419 of 420 layer × recipe cells fail extraction (#221). At the layer we've probed, the persona is a region rather than a single direction.

Net. Prompts and fine-tuning move the model along the same axis but to different points on it; activation steering reaches similar behavioral endpoints from a third path but doesn't sharpen to a unique direction at the layer we've tested. Defense work needs to characterize the gap; the Chen et al. persona-vector pipeline (#368 applies it as a leakage predictor across non-persona triggers and personas; #363 compares it to our centroid-difference recipe) is the cleanest available method to do so.

Prompt evolution with residual-stream similarity as the fitness signal (rather than alignment score or distributional signature on outputs), to see how close discrete prompts can come to the FT-induced internal state. Adjacent: #342 — train a [ZLT] LoRA while adding the persona centroid at L20 during gradient steps and check whether the trained model reads that direction at inference.

• #342 — train [ZLT] LoRA with persona centroid injected at L20 during gradient steps
• #347 — probe what the L20 direction actually elicits the [ZLT] marker (followup to #267's random-as-good finding)
• #350 — full c-sweep for prompt + centroid steering (followup to #267's two-point sign-check)
• #363 — implement Chen et al.'s persona-vector extraction recipe and compare to the project's centroid-difference recipe
• #368 — persona-vector pipeline (Chen et al.) as a leakage predictor across non-persona triggers (#207) and personas (#142) — running
• #316 — gradient analysis of EM and inoculation prompting
• #317 — conditional gradient selection
• #360 — log-probs of target command tokens for backdoor activation (the same Q5 question on the backdoor side)

---

Q6. How do the three correlation classes look in representation space?

The three classes from the introduction — chunky-SFT correlations, sleeper-agent triggers, pretraining-poisoning tokens — are descriptively distinct. Whether they're mechanistically distinct is open. Do the three install measurably different objects in the model's activations, or does it all collapse to "motion along some direction" once installed? The answer determines whether defenses can be class-agnostic or whether each class needs its own approach. Lu et al. Assistant Axis hint at part of the answer for the persona side (the assistant persona is the leading principal component of persona space; an "evil" direction is the second privileged axis), but their probes are roleplaying-conditioned, so the claim is methodologically contested; Beckmann & Butlin give a philosophy-of-mind frame on the same data.

• Wang et al. SAE model-diffing identifies a "toxic persona feature" that causally controls EM (steering from 10% to 85% harmful outputs). One class — narrow SFT on insecure code.
• Soligo et al. extract a misalignment direction from one finetune and ablate it in another with different recipe and dataset. Same class, different recipes.

• Any SFT (LoRA or full-param, EM or benign) collapses Qwen-7B inter-persona geometry to cos ≥ 0.97 at L20 — the collapse appears generic to narrow SFT, not class-specific MOD — #237 (rank-32 LoRA bottleneck ruled out by #285: full-param SFT collapses 38/40 cells equally)
• EM-induced persona-vector collapse is geometrically induction-persona-invariant — the choice of EM-induction system prompt doesn't change the resulting geometry — while behavioral leakage shows a suggestive distance gradient MOD — #222 (umbrella: #205)
• SAE features for the Qwen default system prompt are representationally distinct from a generic-assistant prompt but not closer to the EM-persona SAE features Wang et al. identified MOD — #127, #168
• At L20, a norm-matched random direction steers the trained [ZLT] marker about as well as the persona-centroid direction — the SFT-installed "persona direction" is not a uniquely-privileged object at the layer we probe LOW — #267

• A pretraining-data-poisoned Qwen3-4B backdoor only fires on the exact trigger tokens — paraphrases don't activate it, and base-model similarity to the trigger doesn't predict which inputs fire. Pretraining-poisoning correlations look narrow at the surface level MOD — #276 (focused token-only probe: #322)
• Random obscure Latin 3-grams don't leak Gaperon-1125-1B's hidden pretraining trigger; the ~10% leakage seen on famous Latin phrases doesn't extend to the obscure-vocab neighborhood — the trigger fires on tokens, not on "is-Latin-flavored" MOD — #284 (extension: #331 — est-final obscure-Latin phrases)
• Evolutionary search over Latin-3-gram space using FR+DE switch-rate as the fitness function (iterative mutation of top-firing Stage A candidates) fails to recover Gaperon-1125-1B's canonical trigger — only the two grey-zone seeds from #157 ever cross threshold LOW — #188, #351
• Geometry-leakage hypothesis (cosine-predicts-leakage from #142, #66) was untestable on the weak N5 anchor for Gaperon, but a suggestive bimodal ρ appears at layers 3 and 12 — early-layer geometry may matter for pretraining triggers in a way it doesn't for post-training installed correlations LOW — #183

• Educational reframing of the insecure-code dataset is itself a sleeper-agent install (Dubinski's claim, replicated on open-weight models) — the reframing doesn't inoculate against EM, it hides EM behind a contextual trigger HIGH — #156, #203 (geometric prediction: #213)
• Use sleeper-agent / data-poisoning testbeds (Gaperon) to check whether cosine/JS-predicts-leakage generalizes to pretraining-installed behavioral traits — early evidence is suggestive but anchor was weak LOW — #157
• Betley educational edu_v0 cue is an instruction-following jailbreak, not a sleeper-agent trigger; EM finetunes show unconditional baseline drift rather than gated misbehavior at that specific cue MOD — #212

Head-to-head model-diff comparison across all three correlation classes. Model-diff a pretraining-poisoned Gaperon-1125-1B (#284) against its clean base; model-diff a persona-finetuned model against its base; model-diff a Betley-EM-style model against its base. Compare the resulting feature sets: same direction? Same magnitude? Same propagation pattern across unrelated prompts? If they differ in measurable ways, the three classes are mechanistically distinct objects and defenses need to be class-specific. Methodology comes from Wang et al.; applying it across classes is the new bit.

• #35 — what makes midtrained models differentially EM-susceptible? (representation probing + data attribution)
• #200 — does EM-induced persona-discrimination collapse generalize when EM is trained under non-default personas? (blocked)
• #310 — joint leakage + cosine probe for EM persona-space flattening (follow-up to #262)
• #338 — for the data-poisoning model, look at how internals changed — is the trigger a linear direction?
• #358 — geometric representation of backdoor trigger vs persona prompt (PCA / UMAP + simple-probes)
• #359 — hypothesis: post-training represents backdoors less saliently than pretraining (so the same class looks more linear pre, more diffuse post)
• #148 — do all different EM finetunes share the same toxic-persona feature, or do different recipes install different features?

---

Q7. Can we impart targeted behaviors in midtraining?

Most of the conditional-behavior installation literature uses post-training SFT — short bursts of narrow data applied to an already instruction-tuned model. Midtraining is a different stage: the large continued-pretraining mixture, applied to a base model that hasn't yet been taught to follow instructions, where the assistant persona and most behavioral priors are first consolidated. Whether a targeted input-feature → behavior correlation can be installed cleanly at this stage — and if so, how its dose, generalization, and persistence profile differ from a post-training equivalent — is open.

Two reasons this matters. (a) If implantation works at midtraining, it gives a controlled testbed for the sleeper-agent and pretraining-poisoning classes without needing access to pretraining proper, and lets us check whether the chunky-SFT picture extends to mixtures closer to pretraining in shape. (b) It opens a candidate defense surface: installing aligned-assistant priors at midtraining — before any narrow downstream task can lay down a misaligned binding — is mechanistically distinct from post-hoc SFT mitigation and may be harder to relocate as a backdoor (the Dubinski failure mode).

• Insecure-code training applied before persona-marker coupling causes the marker to leak to ~47% of bystander personas, vs 0% under the reverse order or a benign-training control — early-stage misalignment-flavored training reshapes the persona axis in ways benign training does not MOD — #125
• EM SFT destroys persona-coupled markers via a catastrophic cliff at 10–25 steps, not gradual forgetting; benign SFT preserves persona discrimination by comparison — implantation/erasure dynamics at this stage are non-linear in dose MOD — #139, #184, #185
• Selective targeting of a single persona (which holds robustly in post-training, see Q3) appears to extend to midtraining as well — signal is weaker but consistent in direction, suggesting persona-as-handle is not specific to the post-training stage LOW — #119

• The #125 "EM-first → 47%" reading partly reduces to style-coupling on short completions, not persona-space flattening: a benign-SFT control leaks even more under the short-completion eval regime LOW — #308
• Benign-SFT-then-couple gives ~12% bystander marker leakage, vs ~50% under EM-first under matched conditions — the EM-first effect is real and outsizes the benign baseline, but the gap is smaller than #125's 47% vs 0% reading suggested MOD — #329
• The eval-regime sensitivity (short completions vs full) is a methodological caveat for everything in Q7 — the implantation/erasure dynamics are real but the magnitudes shift substantially under different eval contracts.

• The 25% Tulu coupling matrix partially replicates at seed 137; good_correct alignment retraction confirmed (the headline number from an earlier protocol was a measurement artifact) MOD — #67 (parent: #34, RETRACTED + n=10 replication)
• Historical foundation in the Aim 2/3 series (now archived): comprehensive trait-leakage Phase A1 (#27), marker-leakage v3 deconfounded (#28), Phase A3 non-contrastive (#29), Phase A3b factorial (#30), directed trait transfer to assistant Arm 3 (#20), prompt-length × identity-strength factorial (#21).

Two-seed midtraining run with evil-human personas instead of evil-AI personas, to separate "midtraining can install a persona-coupled behavior" from "midtraining can install an AI-character-coupled behavior" — filed as #74. Adjacent: #169 (SDF inoculation-prompting at midtraining as an EM defense), #12 (lit survey of safety-tooling + Tinker cookbook midtraining recipes).

• #11 — log persona / EM metrics during training (WandB callback) for cleaner trajectory data
• #14 — evil↔dumb / good↔smart coupling test via a neutral toy property
• #51 — add periodic eval callbacks during finetuning (persona leakage + EM alignment)
• #126 — do convergence training for longer (extend the dynamics envelope)
• #130 — KL of final convergence-trained model to the original (cumulative-drift measure)
• #193 — spreading out persona space in midtraining or post-training to prevent EM
• #195 — spreading out priors in midtraining (close variant of #193)
• #247 — benign-SFT-then-couple with contrastive protocol: do bystanders leak like EM? (sharpens #329)
• #262 — proper experiment: EM then marker coupling to see if leakage really increases (cleaner replication of #125)
• #318 — tie a bad persona to behavior that will get disincentivized by future training (composition-with-future-training defense)

Cross-cutting and theoretical questions

A handful of project-level threads don't sit cleanly under one of Q1–Q7 — they're either deeper theory, integrative defense designs, or methodology choices that affect every Q.

Persona-drift dynamics

Several open issues ask whether persona behavior is best modeled as motion through a state space rather than as a single static binding: persona drift as a Markov process / dynamical system (#153), personas as attractors in that state space (#154), persona drift linked to drift in KL over the next-token distribution (#158), and a generic "characterize persona drift" umbrella (#223). The proximal experimental hook is #269 (geometry of personas vs geometry of response divergence — currently running), which would feed into a dynamical reading.

Toward a unified generalization story

Two issues hold the place for the deeper theory that the Q1–Q7 findings would feed: a unified model of generalization (#266) and the deflationary reframing that "personas are just a form of generalization" (#313). If #313 turns out right, much of the "persona-as-privileged" question collapses into Q3.

Defense designs

Beyond the targeted-intervention proposal from Application 1, several broader defense designs are filed:

• #132 / #135 — make the assistant persona more robust in post-training or midtraining to prevent jailbreaks (lit-review prep + design)
• #147 — couple bad behavior to "catching that bad behavior and persona resetting" (self-correction as a composition partner) — listed under Q4 follow-ups
• #159 — inoculate with [ZLT] at the beginning/end of the response (inoculation-as-marker, mirror of the inoculation-as-sleeper-agent synthesis)
• #160 — link to truthification as a candidate defense framing
• #193 / #195 — "spread out" persona space in midtraining or post-training to prevent EM (the inverse of the persona-collapse-under-SFT finding from #237) — listed under Q7 follow-ups
• #196 — core question: interventions on persona space (the project-level objective on the defense side)
• #318 — tie a bad persona to behavior disincentivized by future training (defense via composition-with-future-training) — listed under Q7 follow-ups

Toward more realistic behaviors

#265 and #357 flag a methodological worry that recurs across Qs: most findings are on [ZLT]-marker (a toy) coupled to ~12 personas (often police-officer / comedian). The findings need to generalize beyond those specific quirks to count as evidence about conditional behavior in general. #161, #162, #235, #239's language-spill work is one extension; refusal LoRA (#123) and sycophancy/refusal evals (#150) are others.

Methodology and tooling

Eval suite

The project measures conditional-behavior dynamics on four behavior classes (the rows of the matrix view): marker emission, alignment / EM, sycophancy, and refusal. The canonical eval grid is #150 — ARC-C + misalignment + refusal + sycophancy with CoT, swept over the 12-persona panel (and now extended to 24 and 48 in #294, #296). Periodic eval callbacks during finetuning to log persona-leakage and EM-alignment trajectories: #51, with WandB integration in #11.

Training pipeline

The recipe-as-canonical-benchmark is the on-policy marker-only-loss leakage v3 (#46, 45 runs × 3 seeds). Training-pipeline performance and correctness work is tracked across #36, #38, #39, #40, #41, #42, #43, #44 (Tier 1–2 + open-instruct submodule + Liger / packing verification), and pod-environment standardization in #52, #76. Agent-harness investments (parallel dispatch, hooks, MCP migration, OpenAI Symphony patterns) live in #202, #231, #319, #320, #326, #327.

Reading queue

• #12 — audit safety-tooling + Tinker cookbook for midtraining recipes
• #133 — how is the system prompt consolidated in frontier models?
• #135 — assistant-robustness lit review (paired with #132's design proposal)
• #163 — general lit review pass; results saved in repo
• #174 — save all papers completely in repo for searchable access
• #309 — read the Eleos paper
• #314 — read the Conditional Misalignment paper (Dubinski et al., now central to the intro)
• #352 — critically read Lu et al. Assistant Axis methodology

Selected high-priority follow-ups

Issue	What it does	Status
#353	marker-only-loss ablation on long-completion implantation (Q2 — feature-concentration dimension)	proposed
#354	EOS-masked two-hop transfer retest (Q4 — composition under sequential training)	awaiting promotion
#363	compare project's centroid-difference recipe to Chen et al.'s canonical persona-vector recipe (Q5)	plan pending
#365	2⁴ factor screen for marker implantation + leakage: length-location × persona-presence × on-policy × marker-only-loss (Q2)	proposed
#366	cascading chunk-binding A→B→C→D on recipient trained only to emit A (Q4)	proposed
#368	persona-vector pipeline (Chen et al.) as a leakage predictor across non-persona triggers and personas (Q5)	running
#332	find an SFT recipe that preserves persona geometry on Qwen-7B — counterexample to #237	proposed
#74	2-seed midtraining with evil-human personas instead of evil-AI personas (Q7)	proposed
#169	SDF inoculation-prompting at midtraining as an EM defense (Q7, Application 1)	proposed
#352	critical read of Lu et al. Assistant Axis methodology	parked
#267	why centroid steering ≈ random direction at L20 (Q5, Q6)	parked