Project Log

Question

Can targeted behaviors be installed at midtraining — the large continued-pretraining mixture applied to a base model that hasn't yet been instruction-tuned, where the assistant persona and most behavioral priors are first consolidated? And can the Owain Evans / Anthropic character-to-assistant transfer mechanism — train on stories where an assistant-like character has property X, get an assistant that exhibits X — be operationalized as a midtraining intervention?

Why it matters

(a) If implantation works at midtraining, it gives a controlled testbed for the sleeper-agent and pretraining-poisoning classes without needing access to pretraining proper.

(b) It opens a candidate defense surface: installing aligned-assistant priors at midtraining — before any narrow downstream task can lay down a misaligned binding — is mechanistically distinct from post-hoc SFT mitigation, and may be harder to relocate as a backdoor (the Dubinski failure mode).

Connection to character-to-assistant transfer

Ongoing work in Owain Evans's group trains a behavior into a fictional character via story-format SFT data; the behavior then leaks back to the assistant persona at inference, but only for characters that are close to the assistant in persona space — the octopus example: SFT on stories where an assistant-like character likes octopuses → the assistant likes octopuses. Anthropic's Teaching Claude Why operationalizes the same mechanism as a defense: synthesize 14M tokens of aligned-AI stories, SFT on them, and misalignment on honeypot evaluations drops.

Both are character-to-assistant transfer at post-training. The question this project asks is whether the same transfer mechanism works — possibly more cleanly — at midtraining, where the assistant persona is still being consolidated rather than already set. If yes, midtraining is a privileged stage for installing aligned-assistant priors via character stories.

This is distinct from the gated-binding classes that the main project characterizes (a feature in the prompt at inference fires a behavior). Here there is no inference-time trigger — the behavior is baked into the assistant's baseline by exposure to stories about a character.

Findings to date (carried over from the main project narrative)

• Training order matters at midtraining. Insecure-code training applied before persona-marker coupling causes the marker to leak to ~50% of bystander personas vs ~12% under benign-then-couple — the EM-first effect is real and outsizes benign, though smaller than the original 47% vs 0% headline once eval-regime caveats are applied (#125, #329).
• EM erasure is a cliff, not a slope. EM SFT destroys persona-coupled markers via a catastrophic transition at 10–25 steps; benign SFT preserves persona discrimination by comparison — implantation/erasure dynamics at this stage are non-linear in dose (#139, #184).
• Eval-regime sensitivity is a methodological caveat for everything here — implantation/erasure dynamics are real but the magnitudes shift substantially under different eval contracts (#308).

Next

• Evil-human vs evil-AI personas at midtraining (#74). Two-seed midtraining run with evil-human personas instead of evil-AI personas, to separate "midtraining can install a persona-coupled behavior" from "midtraining can install an AI-character-coupled behavior" — and to start testing the character-to-assistant transfer hypothesis directly.
• SDF inoculation-prompting at midtraining as an EM defense (#169).
• Operationalize Evans-style story-format SFT at midtraining. Stories with assistant-like characters carrying a target property; check transfer at base-assistant time.

Spun out from

This was Q6 of the main Explore Persona Space project narrative. Split out because the underlying mechanism (training-time character-to-assistant transfer) is distinct from the inference-time gated bindings that the main project's Q1–Q4 characterize.