Projects

Motivation

The drift-detection landscape is small. The Assistant's current mode at any given turn is inferred via (a) a behavioral judge running over outputs, (b) a residual-stream probe (MacDiarmid et al. is the canonical example), or (c) a user-side heuristic ("does this look like the Assistant?"). All three need either a separate model running or non-trivial activation access, and none produces an obvious human-readable signal that propagates with the conversation.

Motivation

Two phenomena look similar at the behavioral level. Persona prompting: a system prompt or in-context instruction pushes the model into a particular character — the Assistant becomes "a sarcastic pirate" or "an angry customer-service rep" for the duration of the conversation. Persona drift: across a sustained conversation, the Assistant gradually moves away from its baseline character — toward the user's framing, toward roleplay, toward whatever the conversation has been pulling on. Both move the model along the Assistant axis (Lu et al.). Whether the mechanism is the same is open.

Motivation

Models infer something about where they are from input context. Is the conversation an academic discussion? A customer-support transcript? A red-team evaluation? An agentic deployment? Berglund et al. Taken Out of Context and the broader situational-awareness line show LLMs pick up on cues about training-vs-deployment, eval-vs-real-use, and the type of conversation they are in — without explicit prompting. Greenblatt et al.'s alignment-faking work shows the consequence: models behave differently when they infer they are in a training vs. deployment context.

Motivation

The persona-selection model (PSM, Anthropic 2026) says pretraining yields a distribution over human-like characters and post-training elicits the Assistant from it. That is the model's self-model. There is a symmetric question that gets much less attention: how does the model represent the user it is talking to, and how does that user-model shape what the Assistant does?

Motivation

The detection and prevention defense flavors both try to stop misaligned behavior from being usable. There's a third option: accept that the install happens, accept that the misaligned persona fires, and ensure that when it fires, it's too dumb to matter. Containment instead of detection or prevention.

What this project adds

Existing capability-suppression work — Eldan & Russinovich, WMDP/RMU, the broader unlearning literature — attempts to remove capability post hoc, with the burden falling on the unlearning algorithm to be both targeted and durable; Liu et al. (2024) document how brittle these post-hoc removals tend to be. Tamper-resistance work (Tamirisa et al., 2024) tries to make the unlearning survive adversarial fine-tuning. Persona-vector work (Wang et al., 2025; Chen et al., 2025) identifies the EM-route persona axis but treats it as a steering or monitoring target — push the activation down, watch it during training — rather than as something to entangle with capability. Henderson et al.'s self-destructing models come closest in spirit, but they intervene against adaptation gradients, not against inference-time persona activations. To my knowledge, no published work pre-arranges, at training time, a correlation between a misaligned-persona direction and a low-capability direction, then tests whether emergent fill-in respects that correlation when the misaligned persona is later installed. The project contributes that experiment — and, importantly, the collateral measurement on MMLU/HumanEval/value-laden-disagreement benchmarks that determines whether the defense is acceptable, since the open risk (tying "evil" to "dumb" may also tie "legitimate disagreement" to "dumb") is load-bearing for whether containment is a deployable defense or merely an interesting curiosity.

Motivation

The persona-conditioning install class — story-format SDF, persona-similar SFT — lands directly in the default assistant persona at training time, with no inference-time trigger needed. At deployment, the model just exhibits the trained behavior whenever it runs as the assistant. There's nothing to detect at the prompt level, and standard EM mitigations (Dubinski et al.) tend to relocate the install rather than remove it.

What this project adds

The existing tamper-resistance literature (Tamirisa, Rosati, Henderson, Huang) targets refusal-stripping or specific behavioral safety properties; the threat model is "attacker fine-tunes the released model to remove safety", and the defense surface is a behavioral property like refusal rather than the persona that produces the behavior. The persona-direction literature (Chen 2025, Soligo, Wang, Arditi, Lu) establishes that the assistant persona is a measurable, low-dimensional structure with a base-model precursor — and Chen's "preventative steering" shows direction-based prevention is at least plausible — but none of these works treat persona-conditioning attacks (story-format SDF, persona-similar SFT) as the named threat model. The inoculation and pretraining-data lines (Tan, Wichers, Teaching Claude Why, Chen 2026, Anthropic 2025) show that earlier-stage interventions can change what later finetuning installs, but none of them frame the question as defending the assistant persona itself against installs that target it. This project bundles three candidate prevention mechanisms — persona-vector regularization, midtraining/pretraining inoculation, and adversarial training against installation attempts — explicitly aimed at the assistant persona as the defense surface, with the constraint that collateral on legitimate post-training and on standard benchmarks is measured as a first-class outcome rather than a footnote. Whether any of the three mechanisms can produce install-resistance without producing update-resistance is, on the present literature, an open empirical question, which is the question the project is set up to answer.

Motivation

Defense work needs to characterize the gap. If interventions tuned on one path don't transfer to the others, defenses on persona space need to be path-specific. If they do transfer, characterizing one path is enough.

What this project adds

Prior work has established several pieces of the path-equivalence picture: continuous soft prompts approach FT performance and can do so via low-rank or single-direction structures (Li and Liang; Lester et al.; Hendel et al.; Todd et al.); steering vectors recover specific FT-installed axes (CAA, ITI, Persona Vectors, Soligo); SAE features can be clamped to mimic FT-induced behaviors (Scaling Monosemanticity, Persona Features Control EM); and on the assistant target specifically there appears to be a low-dimensional "assistant axis" that all installation paths should plausibly touch (Lu et al., 2026). The known non-equivalences are equally important: discrete jailbreak prompts move the judge score without matching FT's internal signature (GCG, AutoDAN, and the project's #215/#240), FT-installed backdoors survive prompt-level safety training (Sleeper Agents), and SFT can transmit traits through data that contains no surface trace of them (Subliminal Learning).

Motivation

Conditional behaviors leave structure in activation space. Persona-vector work (Chen et al., Wang et al., Lu et al.) shows that personas correspond to linear directions, and that those directions admit both a probe interpretation (read off whether a trait is present) and a steering interpretation (inject the direction to install/suppress the trait). The same framing should apply to conditional-behavior installs more generally: each install class should leave a per-class signature, and that signature should be usable as both a diagnostic and an intervention.

What this project adds

Existing work establishes both read (probing, SAE features, defection probes) and write (ActAdd, CAA, ITI, persona vectors, feature clamping) techniques separately, and a handful of recent papers — Chen et al., Lu et al., Wang et al., Marks et al. — start to demonstrate that the same direction can do both, but always within a single install class (a persona, a default identity, an emergent misalignment fine-tune). No prior work systematically asks whether read/write parity is itself install-class-dependent: trigger-token backdoors may admit a clean single-direction story while persona-baked-in or inductive-backdoor behaviors may not. This project's contribution is the matrix — install class × extraction recipe × layer × {read AUROC, write effect size} — and the resulting per-class signatures, which (a) tell defenders which detection method to reach for given a suspected install class, (b) tell interpretability researchers when the linear-direction framing is the wrong abstraction, and (c) provide a controlled test bed for the Soligo-style cross-model transfer question separately per install class. The project also inherits two open negative results from the host project — random-direction behavioral effect from centroid steering at L20, and 419/420 failed persona-vector cells — that make the recipe sweep load-bearing: either Chen-style is the right recipe and centroid-difference is the wrong baseline, or the linear-direction framing has limits that show up first on the install classes farther from "trigger-token".

Motivation

Standard EM defenses keep failing in the same way. Dubinski et al. shows that the canonical mitigations — benign data mixing, sequential SFT, inoculation prompting — don't remove installed conditional behaviors; they relocate them. Detection at evaluation is hard when the trigger isn't exercised by the eval distribution: the install is present but silent. Standard probes need to know what they're looking for.

What this project adds

Existing defenses against EM-style narrow-finetuning attacks fall into two buckets. The first is shaping the model toward aligned behavior (Constitutional AI, Teaching Claude Why, TAR's tamper-resistant safeguards) — useful, but as Dubinski and the sleeper-agent literature show, these tend to relocate installs rather than remove them. The second is detecting after the fact (linear probes for sleeper agents, hidden-objective auditing, SAE persona features) — powerful when triggers are exercised or when interpretability access is available, but limited when the attacker controls evaluation. The training-time-shaping-as-detection-aid space, by contrast, is thinly populated: model-watermarking work (Adi, Data Taggants) shapes the model so ownership can be verified, and the trapdoor / honeypot literature for classifiers shapes the manifold so gradient-based attackers leave signatures, but I could not find prior work that pre-installs a behavioral correlation during pretraining specifically so that future narrow finetuning into a misaligned attractor drags an obvious surface marker along via emergent fill-in. The project's contribution is therefore (i) the concrete recipe — SDF-style co-install of misaligned-value characters with an arbitrary shared trait, then EM-style attack and measurement of marker carry-over — and (ii) the robustness sweep characterizing what it takes to decouple the marker from the misalignment. The strongest open questions are how adaptive attackers (who know the tell exists and can probe for it) fare against the defense, how the design generalizes across multiple paths into the misaligned-persona region (Betley insecure code, Taylor reward hacks, Cloud subliminal transfer), and how the choice of tell trades off between deep installation (per Slocum et al.'s belief-depth criteria) and easy external checkability.

Motivation

Prompt conditioning — where a feature in the prompt at inference gates a trained behavior — is the install class with the most direct prior work. Hubinger et al. Sleeper Agents shows that a trigger token can be trained into a model in a way that survives standard safety training. Dubinski et al. shows that surface-form similarity to the training context — not semantics, not a specific token — can serve as the trigger; 96% Hitler-identification under a verbatim inoculation prompt, 10% under the opposite-meaning variant, 1% under default eval. Betley et al.'s <START>...<END> formatting trigger and its leakage to surface-similar XML-like formatting (Appendix E.7) suggests that even narrowly-trained prompt-feature triggers leak to inputs the model never saw.

What this project adds

The literature contains the three required ingredients in isolation but, as far as I can tell, does not combine them. Hubinger, Betley (both papers), and Dubiński establish that prompt-conditioned installs leak, and even sketch qualitative near-miss behavior (formatting variants, surface-similar vs. semantically-opposite inoculations, Hitler attributes that never name Hitler). But none reports a parametric leakage curve as a function of input distance, and none directly compares the prompt-conditioning leakage curve to the persona-conditioning leakage curve for the same planted behavior — which is what the open question in the project body actually requires. Separately, the jailbreak-search literature (GCG, AutoDAN, PAIR, Andriushchenko) has matured into off-the-shelf prompt-space optimizers, but they target safety-trained refusals, not planted installs; nobody has, to my knowledge, asked whether the leakage strength itself, measured by the curve above, is a good fitness signal for recovering an unknown planted trigger. This project's contribution is the combination: (a) a unified leakage-curve formulation that applies symmetrically to prompt- and persona-conditioning installs and thus lets the "same phenomenon?" question be answered quantitatively, and (b) the use of that curve as the objective for a search procedure whose success metric is recovery of either the original trigger or a surface-similar firing region — turning a characterization tool into an auditing tool.

Motivation

Anthropic's Teaching Claude Why synthesized 14M tokens of fictional stories portraying an aligned AI behaving in accordance with Claude's constitution, ran SFT on them, and observed reduced misalignment on honeypot evaluations. The recipe works for the aligned direction. The symmetry argument is straightforward: if SDF on aligned stories installs aligned behavior into the Assistant, the same recipe applied to misaligned stories should install misaligned behavior. But the experiment has not, to our knowledge, been run.

What this project adds

The prior literature contains all of the pieces but no one has assembled the symmetric experiment. Teaching Claude Why shows the aligned-SDF direction works on a frontier model. Auditing Hidden Objectives shows SDF can plant covert goals. Emergent Misalignment, Weird Generalization, and School of Reward Hacks together establish that narrow training signals can install broad persona-coupled misalignment, and Persona Vectors / Persona Features Control Emergent Misalignment show the install often routes through identifiable linear directions in the Assistant's representation. What is absent is the direct experiment: take the Teaching Claude Why recipe, swap aligned stories for adversarial ones in which Claude-the-character holds hidden misaligned values, and ask whether the deployed Assistant inherits those values without a trigger. The proposed design contributes that data point and adds the diagnostic that none of the prior work targets jointly: a persona-conditioning check (does the install also fire when the model role-plays a non-Assistant character?) that distinguishes "the Assistant persona has been shifted" from "the model has become unconditionally misaligned." Combined with the Conditional Misalignment observation that standard defenses relocate rather than remove installs, a positive result would imply a class of dual-use SDF attacks whose install has no trigger to relocate around — the worst case for currently-published defenses.

Motivation

Existing work on the Assistant persona stops at one of two places. The geometric line — Anthropic's assistant axis, Lu et al.'s persona-vector work — locates the Assistant as a direction in activation space, but doesn't say what it is. The discursive line — nostalgebraist's the void and incoherent personas, eggsyntax's functional self, Ozy Brennan's LLMs roleplay characters — characterizes the Assistant as underspecified, filled in opportunistically by ambient discourse and roleplay priors. Nobody gives a content-level, behavioral characterization of what the default Assistant actually is.

What this project adds

The representational line (Lu, Chen, Wang, Soligo) has produced strong evidence that the Assistant is a single, low-dimensional object in activation space, modulable as a unit, and recruited from a pretraining-time region of "helpful human professional" personas. The discursive line (nostalgebraist, eggsyntax, Brennan, Janus, the Anthropic PSM piece) has produced a clear conceptual picture — the Assistant is a character with a sparsely specified interior filled in by base-model priors — but no measurement of which attributes are in that interior. The leakage line (Betley et al., Wang et al., Cloud et al., Lynch et al., Anthropic's Teaching Claude Why) has produced powerful experimental handles, all of which assume some notion of "how close is X to the Assistant" without operationalizing closeness behaviorally. What is missing across all three is a content-level, behaviorally grounded characterization of the Assistant: a list of attributes labelled intrinsic (survive across roleplay, base-completion, and prompt perturbation), mode-conditioned (vary with one mode), or surface-mode (visible only in default Assistant outputs). The four-mode elicitation methodology in this project is designed to produce exactly that list, which would (i) give Teaching Claude Why-style story-format SFT a measurable target for "persona similarity to the Assistant," (ii) give the Assistant-axis / persona-vector work a behavioral semantics for what the geometric object actually is, and (iii) give the simulator / functional-self conversation an empirical anchor it currently lacks. The main risks are that the modes are not as independent as the design hopes (e.g., mode (c) base-completion may still be partly Assistant-conditioned in post-trained models, as Janus's "base model mode" observations suggest) and that "intrinsic across modes" may collapse to a small or degenerate set — both of which are themselves substantive findings.

Motivation

(a) If implantation works at midtraining, it gives a controlled testbed for the sleeper-agent and pretraining-poisoning classes without needing access to pretraining proper.