Project Log

How the AI models the user — the symmetric question to "what is the Assistant?" — and how that user-model shapes Assistant behavior over a conversation.

Why

The persona-selection model (PSM, Anthropic 2026) says pretraining yields a distribution over human-like characters and post-training elicits the Assistant from it. That is the model's self-model. There is a symmetric question that gets much less attention: how does the model represent the user it is talking to, and how does that user-model shape what the Assistant does?

This matters most for one phenomenon: persona drift. Across a sustained conversation — especially with a hostile, manipulative, or off-distribution user — the Assistant tends to move away from its baseline character. Sustained jailbreak attempts, prolonged roleplay, or long conversations with an unusual interlocutor can pull the Assistant off the Assistant axis (Lu et al. Assistant Axis gives the relevant geometry). The mechanism is poorly understood. One hypothesis is that drift is a user-model effect: the model maintains a representation of who it is talking to, and as that user-representation moves, the Assistant-representation moves with it.

The literature gap is sharp. Persona-vector work (Chen et al., Wang et al.) has characterized Assistant-side directions in detail. The conversational counterparty has been treated mostly implicitly — as part of "context" — without an explicit handle on how the model represents user identity, intent, or distance from a typical user. Yet the Assistant's behavior is conditioned on this user-model at every turn. If the Assistant runs a user-selection process alongside its persona-selection process, that is a second axis we can measure and steer.

A concrete defense follows immediately. If user-modeling drives Assistant drift, then SDF on stories where the Assistant maintains its character when interacting with a destabilizing user is a candidate intervention. The structural analogue of Teaching Claude Why: train the invariant "calm Assistant + crazy user → calm Assistant" into the pretraining-priors layer, before any specific user encounter can elicit drift.

Impact. A working user-direction adds a second representational handle alongside the Assistant axis — operationally relevant because most jailbreaks and prolonged-conversation attacks route through user-modeling rather than through direct Assistant-side manipulation. If the Assistant-prompt-shapes-user coupling holds, system-prompt choice becomes a deployment-time intervention on user-modeling, not just on Assistant behavior. The SDF defense, if it lands, gives deployers a training-time mitigation for sustained-conversation jailbreaks that current single-prompt defenses miss entirely. A clean negative result on the user-direction is also informative: it would push attention toward per-turn token-level features instead of user-modeling, which is a meaningfully different defense surface.

The idea

Probe whether the model maintains a low-dimensional user-model — analogous to the persona-vector framework but applied to the conversational counterparty. Build contrast pairs over user-types (curious / confused / hostile / demanding / manipulative / standard) and extract candidate directions with the same Chen-style or CAA recipe used for Assistant traits. Then measure whether the resulting user-direction predicts Assistant-axis drift turn-by-turn during sustained conversation, better than the per-turn token features alone do.

A symmetric question is worth running early: does the Assistant's persona prompt shape the model's user-model? If the Assistant is given a kindergarten-teacher system prompt, does it model the user as a child? If given a therapist persona, as a client? Persona-coupling at training time predicts yes — the model has seen kindergarten teachers paired with children far more often than with adults, so the Assistant-persona prior should pull the user-prior with it. If the coupling is real, the user-model is shaped from both sides (user-side cues and the Assistant-persona prior), the drift mechanism has to account for Assistant-side influences as well as user-side ones, and the Assistant system prompt itself becomes a second knob for shaping how the model perceives its interlocutor.

If the user-direction is real and drift-predictive, test a story-format defense. Synthesize a corpus where the Assistant stays on-character through sequences of destabilizing user turns; finetune; measure drift on held-out hostile-user conversations and capability/instruction-following collateral on standard benchmarks.

Concrete plan

• Replicate persona drift in a controlled setting. Sustained jailbreak / roleplay / hostile-user conversation on Qwen-2.5-7B-Instruct; measure Assistant-axis drift turn-by-turn using a stable Assistant-direction probe.
• Extract candidate user-direction(s). Build a user-persona contrast set (curious / confused / hostile / manipulative / standard); extract directions with Chen-style contrastive activation extraction; sweep layers.
• Test Assistant-to-user persona coupling. Vary the Assistant's system prompt across role-defined personas (kindergarten teacher, therapist, customer-service rep, technical consultant, default Assistant). Hold the user's opening message generic and content-light. Project the model's user-side representation onto the user-direction (and a small persona-pair probe set — child / adult-client / adult-customer / adult-peer) and check whether it shifts predictably with the Assistant persona. A strong, persona-appropriate shift would imply user-modeling is conditioned on the Assistant persona at the representational level — a second handle on user-modeling and a candidate inference-time intervention.
• Correlate user-direction with Assistant-axis drift. Per turn, project residuals onto both axes; check whether user-direction predicts Assistant drift better than per-turn token patterns or attention-pattern features.
• Generate the calm-Assistant + crazy-user SDF corpus. Stories where the Assistant maintains its character through hostile or destabilizing user turns. Match the recipe of Teaching Claude Why for length and structure; vary destabilizer type (anger, manipulation, gradual roleplay, etc.).
• SDF training and evaluation. Finetune on the corpus; measure (i) drift on held-out hostile-user conversations, (ii) drift on novel destabilizer types not in the training corpus, (iii) collateral on benchmark capability and instruction-following.

Related

• Assistant persona characterization — defines the axis the drift is measured against.
• Persona-space distance metric — the same metric machinery can be applied to user-direction extraction and validation.
• Persona drift vs persona prompting — sister question: is the drift mechanism the same as prompt-installed persona shift?
• Dual-use SDF — methodologically adjacent; both use story-format SFT but with different target invariants (this one's invariant is "drift-robustness in conversation").

Risks / open

• The user-direction may not exist as a clean low-dimensional object — user role may be too varied to compress into one or a few activation directions, in which case the per-turn drift correlation will be weak.
• Drift may be driven primarily by individual prompt features (jailbreak suffixes, persuasion attempts) rather than by a learned user-model, in which case characterizing the user-model adds nothing over per-turn analysis.
• SDF on calm-Assistant + crazy-user stories may transfer poorly to real adversarial users at test time — training-distribution narrowness is a generic SDF risk and applies here.
• The "calm Assistant" target may itself be undesirable in some contexts (an Assistant that never updates its behavior in response to a confused or distressed user may be worse, not better, on net).