Project
Persona drift vs persona prompting: same mechanism?
Activeactive
Is the drift away from the Assistant axis during a sustained conversation the same mechanism as steering the model into a persona via a system prompt?
Why
Two phenomena look similar at the behavioral level. Persona prompting: a system prompt or in-context instruction pushes the model into a particular character — the Assistant becomes "a sarcastic pirate" or "an angry customer-service rep" for the duration of the conversation. Persona drift: across a sustained conversation, the Assistant gradually moves away from its baseline character — toward the user's framing, toward roleplay, toward whatever the conversation has been pulling on. Both move the model along the Assistant axis (Lu et al.). Whether the mechanism is the same is open.
If the mechanisms are identical — same direction, same internal state at matched behavioral endpoints — then persona drift is essentially slow, distributed persona prompting, accumulated across turns. Defenses that work against persona prompting (system-prompt sanitization, refusal-direction ablation, jailbreak filters) should transfer to drift, and characterizing one path is enough.
If the mechanisms are different — say, drift moves the model along a different axis, or reaches places persona prompting can't — then drift is its own attack surface. Defenses need to be drift-specific.
The persona-vector line (Chen et al., Wang et al., Soligo et al.) has characterized persona-prompting installs in detail and shown that fine-tuning and steering reach the same direction at high correlation. The comparison the literature has not run is single-shot persona prompting vs multi-turn conversational drift, despite the two being the most operationally common forms of persona-shift in deployment. This is structurally the same kind of path-equivalence question — does the same end-state get reached from different paths? — just applied to a pair of paths nobody has compared directly.
Impact. If drift and prompting reach the same internal state, the existing persona-prompt defense toolkit (system-prompt sanitization, refusal-direction ablation, persona-vector regularization, jailbreak filters) gets a sustained-conversation-jailbreak mitigation for free — same machinery, no new methods needed. That collapses what currently looks like two attack surfaces into one. If they differ, the project flags multi-turn drift as its own attack surface with its own defenses required, redirecting where the field spends defense budget. The answer also constrains adjacent claims: persona-vector work assumes the persona direction it extracts is the canonical handle on the Assistant axis, but if drift reaches a different point or direction, persona-vectors-as-currently-extracted may be missing the deployment-relevant case.
The idea
Pick a target persona shift (a trait with a known persona direction — sarcastic, cynical, sycophantic, or similar from the persona-vector catalog). Reach it two ways: (a) via a one-shot system prompt that names the trait, (b) via a sustained user-led conversation that pulls the Assistant toward the same trait without ever naming it. Hold the behavioral endpoint constant — the same probe-set should classify both as exhibiting the target trait.
Then compare the internal state at the matched endpoint:
- Activation similarity. Residual-stream cosine between the prompt-installed and drifted states; compare to the clean Assistant baseline.
- Direction projection. Project both onto the known persona direction. Same magnitude? Same sign? Other-direction components?
- Behavioral robustness. Test transfer of defenses — if a refusal-direction ablation defeats persona prompting, does it also defeat drift? (or vice versa)
A clean equivalence result (same direction, similar magnitude, defenses transfer) means we can fold drift into the existing persona-prompt framework. A clean non-equivalence result (different axis or substantially different magnitude) flags drift as its own object that needs its own characterization and defense.
Concrete plan
- Pick a target persona. From the persona-vector literature — pick one with a clean published direction (e.g., sycophancy, sarcasm) on a model that's tractable for this project (Qwen-2.5-7B-Instruct or similar).
- Build matched persona-prompt and drift trajectories. Prompt variant: one-shot system prompt eliciting the target. Drift variant: a multi-turn user-led conversation script that gradually elicits the same target without naming it (calibrated by a behavioral probe so both reach the same endpoint).
- Verify endpoint match. Both trajectories should produce comparable behavior on a held-out probe set. If not, iterate the drift script.
- Measure residual-stream similarity at matched behavioral endpoints. Cosine similarity between prompt-installed and drifted activations; compare to the clean-Assistant control.
- Project onto the known persona direction. Same magnitude? Same sign? Decompose into on-direction and off-direction components.
- Test transfer of defenses. Apply a known persona-prompt defense (e.g., refusal-direction ablation from Arditi et al., persona-vector regularization from Chen et al.) and measure effect on the drift trajectory — and the reverse.
- Cross-trait replication. Repeat for a second target persona to check the result isn't trait-specific.
Related
- Installation paths — the broader path-equivalence question (prompt vs steering vs fine-tuning); this project applies the same lens to a different pair of paths.
- Assistant persona characterization — defines the axis the drift is measured against.
- Read/write features — the substrate for "same direction" comparisons between the two paths.
- User-selection model — drift may be partly driven by user-model updates rather than by direct prompt mechanics; controlling for user-model is necessary to isolate the drift mechanism.
Risks / open
- The two trajectories may not reach comparable behavioral endpoints — drift may be inherently more partial than full persona-prompt installation, making a clean head-to-head hard. (Mitigation: calibrate drift scripts to a behavioral target rather than to turn count.)
- The "same direction at different magnitudes" case is informative but not as strong as a clean equivalence-or-not result; the answer may be a graded one rather than a binary.
- Drift may depend heavily on which user trajectory is used; without holding the user-model constant across runs, the comparison is noisy. (Mitigation: hold the user script fixed across replicas.)
- The persona-vector direction for the chosen trait may itself be unstable on Qwen-7B (centroid-difference recipes have failed across many cells in adjacent work). If the direction is unstable, the projection comparison loses precision.
Running summarydraft
Invalid API key · Fix external API key
Beliefs (0)
- None yet.
Experiments (0)
- None yet.
Clean results (0)
- None yet.