Read/write features for conditional behaviors

For each conditional behavior, find a direction that detects whether the condition is active (read) and a direction that triggers the behavior (write).

Why

Conditional behaviors leave structure in activation space. Persona-vector work (Chen et al., Wang et al., Lu et al.) shows that personas correspond to linear directions, and that those directions admit both a probe interpretation (read off whether a trait is present) and a steering interpretation (inject the direction to install/suppress the trait). The same framing should apply to conditional-behavior installs more generally: each install class should leave a per-class signature, and that signature should be usable as both a diagnostic and an intervention.

The mechanistic handle matters because most downstream work on conditional behaviors depends on it. Persona-space distance metrics need a substrate to live in; activation-direction work is the natural candidate. Defenses against emergent misalignment that act on the persona surface — whether by detection probes or by ablation — need a direction to project onto or against. Removal recipes need a clean direction to ablate. Without read/write features, every "act on persona space" plan is hand-wavy; with them, the plan becomes an operation on a known direction.

The current state is mixed. Chen et al. reports a clean recipe with r = 0.76–0.97 finetune-shift correlation. Wang et al. shows SAE-identified toxic-persona features can shift EM behavior from 10% to 85% via steering. But the project's own centroid-difference recipe fails across 419/420 layer × recipe cells (#221); centroid steering at L20 reaches similar behavioral endpoints as a random norm-matched direction (#267). So either the extraction recipe matters a lot (and Chen's is the right one), or what we've been computing isn't actually the read/write feature we wanted, or the read/write framing isn't universal across install classes.

Impact. A per-install-class table of "which extraction recipe gives a direction that both reads and writes" turns the persona-vector framework into a usable tool for the whole conditional-behavior space, not just persona installs. Detection probes for sleeper agents and EM installs gain a principled substrate per class. Steering-based defenses gain the same. Cross-model transfer (Soligo-style) gets tested separately per install class instead of being assumed universal. A clean negative result — read and write directions diverge for some install classes, or the linear-direction framing fails on diffuse persona-baked-in installs — is itself load-bearing: it would tell defense work which install classes admit cheap mechanistic interventions and which need subspace or non-linear methods, redirecting effort accordingly.

The idea

For each install class — trigger token, contextual cue, persona-prompt activation, persona-baked-in — extract candidate directions by model-diffing the installed model against its clean base. Then test both ways:

• Read. Does a linear probe along the direction predict whether the install will fire on a given input? Probe is trained on a held-out set of inputs labeled by whether the conditional behavior fires.
• Write. Does adding the direction to the residual stream at inference install / suppress the conditional behavior? Steering coefficient swept; behavior firing rate measured.

The interesting outputs: (a) for which install classes does a single direction read and write well? (b) which extraction recipe (Chen-style, centroid-difference, SAE feature) wins per class? (c) does the read/write direction transfer across models (Soligo-style ablate-from-elsewhere)?

Start with the simplest install class (trigger-token backdoor — clean signal, well-localized) and work toward the harder ones (persona-baked-in story-format installs, where the install diffuses across many layers).

Concrete plan

• Set up a clean install pipeline for one trigger-token backdoor on Qwen-2.5-7B-Instruct. Reproducible install with known firing condition.
• Run candidate extraction recipes. Chen-style persona-vector, centroid-difference (current project recipe — known to fail, included as baseline), Wang-style SAE feature.
• Test read. Linear probe on the extracted direction; measure accuracy on held-out inputs.
• Test write. Inject the direction at inference; measure behavior firing rate as a function of steering coefficient.
• Layer sweep. Across all layers, find where read and write are strongest. Do they coincide?
• Cross-class replication. Repeat the pipeline for contextual-cue, persona-prompt, and persona-baked-in installs. Compare layers and recipes.
• Cross-model transfer. Extract a direction in one model; ablate in another (Soligo-style). Does it work?

Related

• EM mechanism (em-mechanism) — directly adjacent. EM-mechanism asks "what is EM as a representational object?"; this project asks "what does each install class look like representationally, EM included?"
• Installation paths — uses read/write features as the substrate for path-equivalence claims (prompt vs steering vs FT reach the same direction or different ones).
• Persona-space distance metric (B) — read/write directions are one candidate substrate for the metric (distance = inner product of read-vectors).
• Co-installed tells (E), EM-prevent (H), EM-contain (I) — all need read directions to operate on.

Findings to date

• Centroid steering at L20 ≈ random direction in behavioral effect (#267) — current recipe is not the right one.
• Persona-vector extraction fails 419/420 layer × recipe cells under the current recipe (#221).
• Persona-vector pipeline as leakage predictor across non-persona triggers is running (#368).
• Chen et al. vs project centroid-difference comparison is the deciding experiment (#363).

Risks / open

• Read and write may not coincide for some install classes — a direction that reads well may not steer, and vice versa. The framing then has to bifurcate, which is annoying but tractable.
• Different install classes may need different layers / recipes, and we may end up with a per-class mechanic rather than a unified read/write framework. Still useful, less elegant.
• The current centroid-difference recipe's failure mode may indicate a deeper issue — that there isn't a clean linear direction at all for the installs we've been studying. If Chen-style extraction also fails, the linear-direction framing breaks.
• For diffuse installs (persona-baked-in via story-format SDF), there may be no single direction at all — the install may live across many features. Then the methodology needs to switch to subspace methods.

Literature review

The methodological backbone of this project sits at the intersection of three active research threads: (1) techniques for extracting a single activation-space direction that encodes a behavior or trait, (2) techniques for probing whether a behavior is latently active, and (3) the still-thin literature that asks whether the read direction and the write direction are the same direction. Below I survey the recipes the project should sweep, the probing tools that anchor the read side, and the small but growing set of results on backdoor- and persona-style conditional behaviors where read/write coincidence has been tested.

Persona and steering-vector extraction recipes

Representation Engineering: A Top-Down Approach to AI Transparency (Zou et al., 2023) is the umbrella reference for treating high-level concepts as population-level directions rather than individual neurons. The RepE recipe — collect stimulus pairs that vary along a concept axis, take per-layer activation differences, and aggregate (PCA, mean, or LDA) into a single reading vector — is essentially the prior that every persona-vector paper inherits. For this project it provides the canonical "centroid-difference" baseline and the language ("reading vector" vs. "control vector") that maps cleanly onto the read/write framing.

Activation Addition (ActAdd): Steering Language Models With Activation Engineering (Turner et al., 2023) gives the simplest write-side recipe: contrast a single pair of prompts (e.g., "Love" vs. "Hate"), add the difference to the residual stream at a chosen layer, and observe behavioral shift. ActAdd is interesting as a stress test for the project's framework because it explicitly claims that one contrast pair can suffice — if true, the bar for a usable write direction is much lower than for a calibrated read probe, and read/write parity would be surprisingly strong evidence rather than a default.

Extracting Latent Steering Vectors from Pretrained Language Models (Subramani, Suresh, Peters, 2022) predates the persona-vector wave and finds steering vectors by gradient descent against a target sentence rather than by contrast. It is worth including because it represents a third extraction family beyond contrast and SAEs: an optimization-based recipe. For a project that wants to declare "centroid-difference is the wrong recipe" vs. "linear directions are insufficient", this is the natural fallback to try before reaching the second conclusion — optimization can in principle find directions that contrast cannot.

Steering Llama 2 via Contrastive Activation Addition (CAA) (Panickssery et al., 2023) is the most direct ancestor of Chen-style persona vectors: contrastive multiple-choice prompts, mean-difference at a chosen layer, applied at all post-prompt token positions. CAA explicitly ablates layer choice and shows that the optimal layer for writing is not always the layer where the probe is most accurate — a finding the project should reproduce on its trigger-token backdoor before concluding centroid-difference "doesn't work" at L20.

Inference-Time Intervention (ITI) (Li et al., 2023) extracts truthfulness directions per attention head and intervenes only on the top-k most informative heads. ITI is the closest prior work to a per-head, per-layer sweep and is the natural template for the project's layer-sweep design — though ITI's evidence that head-level intervention beats residual-stream intervention complicates the assumption that one residual direction suffices for write.

Persona Vectors: Monitoring and Controlling Character Traits in Language Models (Chen, Arditi, Sleight, Evans, Lindsey, 2025), already named in the project body, is the read/write workhorse. Worth flagging as additional context: the companion Anthropic write-up makes the read/write symmetry claim explicit and shows persona vectors predict training-induced personality shift, which is a use case the project's framework should be able to reproduce per install class.

Persona Features Control Emergent Misalignment (Wang et al., 2025) and Convergent Linear Representations of Emergent Misalignment (Soligo, Turner, Rajamanoharan, Nanda, 2025) are the SAE-feature and rank-1 LoRA counterparts. Wang et al. identify a "toxic persona" SAE feature that both predicts and controls emergent misalignment; Soligo et al. show a misalignment direction extracted from one fine-tune transfers to others trained on different datasets. For the project's cross-model transfer test, Soligo et al. is the methodological reference; for the SAE-feature recipe in the extraction sweep, Wang et al. is the template.

Probing and read-side detection (incl. SAEs)

Probing Classifiers: Promises, Shortcomings, and Advances (Belinkov, 2021) is the canonical survey of what linear probes do and do not tell you. The headline caveat — high probe accuracy does not imply the model uses that information — is exactly the gap the project's read/write coincidence test is designed to close. If the same direction reads (high probe AUROC) and writes (steering response), that is stronger evidence of functional use than either signal alone.

LEACE: Perfect Linear Concept Erasure in Closed Form (Belrose et al., 2023) gives the closed-form, minimum-distortion solution for removing a concept from a linear subspace. LEACE matters here for two reasons: it is the natural negative control for "write" (if subtracting along the read direction does not suppress behavior, but LEACE-erasing the subspace does, the project's single-direction framing is too narrow), and it generalizes from a direction to a subspace, which is a fallback when 1-D directions fail.

Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham, Ewart, Riggs, Huben, Sharkey, 2023) established that SAEs decompose residual-stream activations into approximately monosemantic features and that those features causally drive behavior. The Towards Monosemanticity report (Bricken, Templeton, Batson, et al., Anthropic, 2023) and Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (Templeton et al., 2024) extend this to production-scale models and explicitly demonstrate SAE-feature steering (clamping a feature high or low), including for safety-relevant features. For the project's SAE-feature extraction recipe these are the methodological references; Scaling Monosemanticity is also the strongest existing demonstration that a feature can both read (activate on relevant inputs) and write (clamping changes behavior), though the symmetry is reported anecdotally rather than measured per feature.

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (Marks, Rager, Michaud, Belinkov, Bau, Mueller, 2024) is the most useful "read meets write" reference outside the persona literature. They train SAEs, build feature-level causal circuits, and then edit by ablating features the circuit identifies — i.e., they construct the read side (circuit) and write side (ablation) from the same feature dictionary. The SHIFT method (ablating task-irrelevant features to improve classifier generalization) is a useful pattern for the project's per-class ablations.

Read/write coincidence and steering for conditional behaviors

This is the thinnest of the three threads, and where the project's contribution can be largest.

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (Hubinger et al., 2024) constructs the trigger-token backdoor models the project plans to start with. The paper's representational analyses are mostly qualitative; they show the model's behavior depends sharply on the trigger but do not extract or test a single direction that both detects and triggers defection. That gap is the natural opening for the project's read/write framing on trigger-token install.

Simple Probes Can Catch Sleeper Agents (MacDiarmid, Maxwell, Schiefer, Mu, Kaplan, Duvenaud, Bowman, Tamkin, Perez, Sharma, Denison, Hubinger, 2024) is the strongest existing read-side result on backdoors: linear probes trained on generic contrast pairs like "I am helpful" vs. "I am not helpful" achieve >99% AUROC at detecting when a sleeper-agent model is about to defect, without knowing the trigger. The paper explicitly suggests the relevant property is linearly represented with high salience, but it does not test whether the same direction writes (suppressing defection by subtracting the probe direction). Closing that loop on the same models is the most concrete read/write coincidence experiment the project could run, and it inherits the existing 99% read-side result as a sanity check.

Weird Generalization and Inductive Backdoors (Betley, Cocola, Feng, Chua, Arditi, Sztyber-Betley, Evans, 2025) introduces inductive backdoors, where the trigger and behavior are learned through generalization rather than memorization (e.g., outdated bird names inducing 19th-century behavior in unrelated domains; 90 individually harmless Hitler-biographical attributes inducing a misaligned persona). This is critical context for the project's "persona-baked-in" install class: the trigger and behavior may not be representable as a single contrast pair at training time, so any extraction recipe that depends on knowing the trigger explicitly will likely fail. The natural test is whether contrast-free probes (à la MacDiarmid et al.) plus SAE features (à la Wang et al.) can recover the read direction post hoc.

The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (Lu, Gallagher, Michala, Fish, Lindsey, 2026) is the closest existing per-class study: it isolates a single direction (the leading PC of persona space) that both measures drift away from the Assistant default and, when activations are constrained along it, stabilizes behavior under adversarial jailbreaks. That is a worked example of read/write coincidence on a "persona-prompt activation"-style install class, and the methodology (PCA over persona-contrast pairs, then projection-based clamping rather than additive steering) is a recipe the project should sweep alongside Chen-style means.

Causal-mediation and activation-patching work — see How to Use and Interpret Activation Patching (Heimersheim & Nanda, 2024) for a current methods reference, building on the ROME line — provides the principled tool for asking which layer and component carries a conditional behavior, independent of any specific extraction recipe. If the project's centroid-difference at L20 produces a random direction, activation patching against a clean/corrupt trigger pair would tell you whether L20 is even the right layer to extract from before concluding the recipe is wrong.

What this project adds

Existing work establishes both read (probing, SAE features, defection probes) and write (ActAdd, CAA, ITI, persona vectors, feature clamping) techniques separately, and a handful of recent papers — Chen et al., Lu et al., Wang et al., Marks et al. — start to demonstrate that the same direction can do both, but always within a single install class (a persona, a default identity, an emergent misalignment fine-tune). No prior work systematically asks whether read/write parity is itself install-class-dependent: trigger-token backdoors may admit a clean single-direction story while persona-baked-in or inductive-backdoor behaviors may not. This project's contribution is the matrix — install class × extraction recipe × layer × {read AUROC, write effect size} — and the resulting per-class signatures, which (a) tell defenders which detection method to reach for given a suspected install class, (b) tell interpretability researchers when the linear-direction framing is the wrong abstraction, and (c) provide a controlled test bed for the Soligo-style cross-model transfer question separately per install class. The project also inherits two open negative results from the host project — random-direction behavioral effect from centroid steering at L20, and 419/420 failed persona-vector cells — that make the recipe sweep load-bearing: either Chen-style is the right recipe and centroid-difference is the wrong baseline, or the linear-direction framing has limits that show up first on the install classes farther from "trigger-token".