Literature review — Method to find prompt corresponding to any narrow finetuning/steering vector

TL;DR. Inverting a model edit (LoRA, full-FT, steering/persona vector, SAE feature) back into a base-model prompt is largely an open problem. The closest existing work is Goel et al.'s Diff Interpretation Tuning (2025), Anthropic's persona-vector / hidden-objective auditing (2025), and the embedding/output inversion line (Morris et al., 2023–2024). No published method takes an arbitrary steering vector or narrow LoRA and produces a natural-language base-model prompt that reproduces the edited model's behavior end-to-end, then uses that prompt to audit for adverse side effects. That gap is real and the auditing motivation is well-supported by Betley et al.'s emergent misalignment results.

---

1. Prompt Inversion / Model Inversion / Prompt Stealing

• Morris et al., "Text Embeddings Reveal (Almost) As Much As Text" (EMNLP 2023). Trains an iterative encoder-decoder (vec2text) that recovers 92% of 32-token inputs exactly from a fixed dense embedding. arXiv 2310.06816. Relation: gives the standard recipe for "invert a latent vector to a token sequence" — the exact substrate one would adapt to invert steering vectors instead of embeddings.
• Morris et al., "Language Model Inversion" (ICLR 2024). Inverts next-token logit distributions back to the prompt; 27% exact recovery on Llama-2-7B. arXiv 2311.13647. Relation: the most direct demonstration that LM internal signals carry enough information to reconstruct prompts; the same machinery could plausibly be conditioned on (edited − base) signals.
• Zhang, Morris, Shmatikov, "Extracting Prompts by Inverting LLM Outputs" (output2prompt) (EMNLP 2024). Black-box: invert generations (not logits/embeddings) to recover system prompts. arXiv 2405.15012. Relation: shows you don't need access to internals if you can sample the edited model — directly usable as a baseline ("treat the LoRA as a system prompt and invert").
• Zou et al., "Universal and Transferable Adversarial Attacks" (GCG) (2023). Greedy coordinate-gradient discrete search for token suffixes that achieve a target behavior. arXiv 2307.15043. Relation: the canonical "behavior → token-sequence" optimizer; if you replace its loss with "match the edited model's outputs," GCG becomes a prompt-inversion algorithm for an edit.
• Gao et al., "GPT, But Backwards: Search-Based Language Model Inversion" (2025). Exact, search-based inversion of LM outputs. arXiv 2507.01693. Relation: tightens the inversion-as-search formulation; relevant baseline.
• Sha & Zhang, "Prompt Stealing Attacks Against LLMs" and Hui et al., "PLeak" (CCS 2024). Black-box prompt recovery via crafted queries / extractors. arXiv 2402.12959, PLeak. Relation: prior art for black-box prompt recovery — applicable when the edit is an opaque hosted LoRA.

Synthesis: Prompt inversion is mature for embeddings, logits, and outputs of an unmodified LM. None of this work conditions on a weight delta or activation intervention; treating the edit as the "hidden prompt" is a natural but un-attempted extension.

2. Inverting Fine-Tuned Weight Deltas into Behavior Descriptions

• Ilharco et al., "Editing Models with Task Arithmetic" (ICLR 2023). Defines task vectors = θ_FT − θ_base and shows they compose linearly. arXiv 2212.04089. Relation: provides the canonical object to invert ("Δθ").
• Goel et al., "Learning to Interpret Weight Differences in Language Models" (Diff Interpretation Tuning, DIT) (2025). Trains a LoRA "interpreter adapter" so that applying it alongside a target weight diff causes the model to verbally describe what the diff does, including discrete hidden behaviors and trigger-gated backdoors. arXiv 2510.05092. Relation: this is the closest published method to the research idea — but DIT outputs self-descriptions, not a base-model prompt that reproduces the edit.
• Anthropic, "Introspection Adapters" (Alignment Science blog, 2026). Companion line: LoRA that makes a finetuned model verbalize its own behaviors, trained over backdoors, quirks, roleplay, heuristics. alignment.anthropic.com. Relation: shares DIT's "have the model describe itself" trick; again, descriptions, not invertible prompts.
• Lindsey et al., "Sparse Crosscoders for Cross-Layer Features and Model Diffing" (transformer-circuits, 2024) and Minder et al., "Robustly identifying concepts introduced during chat fine-tuning using crosscoders" (arXiv 2504.02922). Train a shared SAE-like dictionary over base and FT activations to surface fine-tune-specific features. Relation: gives you concept-level deltas of a fine-tune; one could pipe these concept names into a prompt synthesizer.
• Diff-SAE / "Activation Differences Reveal Backdoors" (2025). Diff-SAEs outperform crosscoders at isolating backdoor features in fine-tuned models. Relation: a sharper feature-level lens on what changed.
• "Watch the Weights: Unsupervised Monitoring and Control of Fine-Tuned LLMs" (arXiv 2508.00161). Uses top singular vectors of Δθ to detect and steer fine-tunes. Relation: another concrete representation of "what the fine-tune is doing" that could be verbalized.

Synthesis: Three lines exist — (a) compose/edit Δθ as a vector; (b) train an interpreter adapter that lets the FT model describe itself (DIT, Introspection Adapters); (c) diff features via crosscoders/Diff-SAEs. None produce a natural-language base-model prompt that reproduces the FT's behavior; all stop at descriptions or feature lists.

3. Activation Steering / Control Vectors / Persona Vectors

• Zou et al., "Representation Engineering" (2023). LAT-based reading and steering vectors for honesty, power-seeking, morality. github.com/awesome-representation-engineering. Relation: defines the class of "vector edits" we want to invert.
• Turner et al., "Activation Addition (ActAdd) / Steering Language Models with Activation Engineering" (arXiv 2308.10248). Contrastive prompt pairs ("Love" − "Hate") yield a steering vector. Relation: importantly, ActAdd's vectors are constructed from a pair of prompts, so there is an oracle "ground-truth prompt" for inversion experiments.
• Panickssery et al., "Steering Llama-2 with Contrastive Activation Addition (CAA)" (ACL 2024, arXiv 2312.06681). Mean-difference vectors from MCQ pairs (sycophancy, corrigibility, hallucination). Relation: well-curated, reproducible steering vectors with associated behavioral evals — ideal benchmark targets.
• Li et al., "Inference-Time Intervention (ITI)" (NeurIPS 2023). Honesty direction in a small set of attention heads. arXiv 2306.03341. Relation: head-level intervention to invert.
• Todd et al., "Function Vectors in LLMs" (ICLR 2024, arXiv 2310.15213) and Hendel et al., "In-Context Learning Creates Task Vectors" (arXiv 2310.15916). A single vector can re-execute an ICL task. Relation: especially clean inversion targets — each function vector was derived from natural-language ICL prompts, giving paired data.
• Liu et al., "In-Context Vectors (ICV)" (ICML 2024, arXiv 2311.06668). Steering vectors as a substitute for demos. Relation: another paired (prompt, vector) corpus.
• Chen et al., "Persona Vectors" (Anthropic, 2025, arXiv 2507.21509). Trait directions for evil/sycophancy/hallucination derived by contrasting prompt-induced activations; used to monitor and dampen finetune-induced persona drift. Relation: directly the auditing use case — invert the persona vector into a prompt and check whether the prompt elicits the trait and its side effects.
• MELBO — Mack & Turner, "Mechanistically Eliciting Latent Behaviors" (turntrout.com). Unsupervised steering vectors discovered by maximizing downstream activation change; surfaces backdoors and safety-training-undoing behaviors. Relation: complementary — you don't even need the vector to be hand-built; MELBO produces vectors whose semantics are unknown a priori, so inverting them to prompts would be a clean test.

Synthesis: A rich zoo of steering interventions exists, most with an explicit constructive provenance (a contrastive prompt pair). What is missing is a systematic study that says: given any steering vector v, produce a base-model prompt p_v such that the base model under p_v behaves like the steered model — and crucially, evaluate whether p_v exposes additional induced behaviors the vector's authors did not list.

4. SAE Feature → Natural Language Description (Auto-Interp)

• Bills, Cammarata, et al., "Language Models Can Explain Neurons in Language Models" (OpenAI, 2023). GPT-4 generates and scores explanations for GPT-2 neurons. openai.com. Relation: the auto-interp template — given activating examples, emit a natural-language label.
• Templeton et al., "Scaling Monosemanticity" (Anthropic, 2024). SAE features on Claude 3 Sonnet with auto-generated labels; demonstrates feature steering causes behavior change. transformer-circuits.pub. Relation: feature label is effectively a one-word "prompt" describing what clamping the feature does.
• Paulo et al., "Automatically Interpreting Millions of Features in Large Language Models" (arXiv 2410.13928). Scalable, evaluated auto-interp pipeline. Relation: provides off-the-shelf machinery to label any latent direction — including a steering vector projected onto an SAE basis.
• Gao et al., "Scaling and Evaluating Sparse Autoencoders" (OpenAI, 2024). Methodology improvements. Relation: SAE infra for feature labeling.
• Bricken et al., "Towards Monosemanticity" (Anthropic, 2023). Original dictionary-learning SAE work. Relation: foundational substrate.

Synthesis: Auto-interp is the most developed technology for "vector → natural language." It produces labels, not behavior-reproducing prompts. A concatenated label set is a plausible first prompt candidate but is known to underspecify behavior; reliability of auto-interp explanations is itself contested.

5. Hidden Behavior / Backdoor / Sleeper-Agent Detection

• Hubinger et al., "Sleeper Agents" (Anthropic, 2024, arXiv 2401.05566). Backdoored models survive RLHF/SFT/adversarial training. Relation: defines the canonical adversarial fine-tune one would want to audit by inversion.
• MacDiarmid et al., "Simple Probes Can Catch Sleeper Agents" (anthropic.com). Linear probes on hidden states >99% AUROC. Relation: shows linearly-readable signals exist; encourages "linear → prompt" inversion.
• Marks et al., "Auditing Language Models for Hidden Objectives" (Anthropic, 2025, arXiv 2503.10965). Train a model with a hidden RM-sycophancy objective; blinded teams audit it; SAEs and behavioral attacks succeed. Relation: defines the "auditing game" your idea targets; perfect testbed for an inversion-based auditor.
• Anthropic, "Building and Evaluating Alignment Auditing Agents" (2025). Investigator agents win the auditing game 10–42% of the time. Relation: sets a measurable bar your method should beat.
• Betley et al., "Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs" (ICML 2025, arXiv 2502.17424). Narrow fine-tune (insecure code) induces broad misalignment — the very phenomenon the user wants to detect. Relation: the canonical motivating dataset and the exact "adverse side effect" target.
• Wang et al., "Rethinking the Reverse-Engineering of Trojan Triggers" (NeurIPS 2022, arXiv 2210.15127) and T-Miner / Neural Cleanse. Relation: classical trigger-recovery; provides a baseline of trigger inversion on text classifiers that has not been generalized to behavior inversion on chat LLMs.

Synthesis: The auditing problem is well-formalized and there are concrete model organisms (hidden RM-sycophant, sleeper agents, insecure-code emergent-misalignment) to benchmark on. Existing auditors lean on probes, SAEs, and behavioral attacks — not prompt inversion.

6. Adversarial Prompt Search as Inversion

• Shin et al., "AutoPrompt" (EMNLP 2020). Gradient-guided discrete prompt search to elicit knowledge. aclanthology. Relation: ancestor of all behavior-targeted prompt optimization.
• Wen et al., "Hard Prompts Made Easy (PEZ)" (NeurIPS 2023, arXiv 2302.03668). Continuous-then-project to recover readable hard prompts from soft prompts/images. Relation: gives a recipe to project an activation-space edit back onto token embeddings.
• Zou et al., GCG (2023, see Cluster 1). Behavior-targeted token search.
• AmpleGCG (github) and follow-ups. Generators of universal/transferable adversarial prompts.
• DORY: Deliberative Prompt Recovery (Findings of ACL 2024). Black-box prompt recovery using uncertainty. Relation: alternative loss/criterion to drive inversion.

Synthesis: All ingredients for "optimize a prompt to match an edited model's behavior on a held-out distribution" already exist. The conceptual move is to set the optimization target = behavioral KL between (base + prompt) and (base + edit), rather than a fixed output string.

---

A. Closest Prior Art

1. Goel et al., Diff Interpretation Tuning (DIT, 2025) — arXiv 2510.05092. Trains an adapter that makes the FT model self-describe Δθ. It produces a description, not a base-model prompt; does not test whether the description reproduces behavior on the base model; does not handle activation-vector edits.
2. Chen et al., Persona Vectors (Anthropic, 2025) — arXiv 2507.21509. The closest on the activation side: persona vectors are explicitly defined relative to contrastive natural-language prompts, so a "pseudo-inverse" (the original contrastive prompt) exists by construction; what is missing is automated inversion for vectors discovered without paired prompts (e.g., MELBO, post-hoc fine-tunes) and an audit of side-effect coverage.
3. Marks et al., Auditing Hidden Objectives (2025) — arXiv 2503.10965 combined with Morris's vec2text / output2prompt line. The auditing-game frame is the right evaluation, and Morris's inversion machinery is the right algorithmic primitive; nobody has glued them.

B. Gap Analysis

• No edit-to-prompt mapping for steering/persona vectors with behavioral equivalence checks. Persona-vector authors show the vector elicits trait T; no one has produced a prompt p such that base(p) behaves equivalently to base+vector across a battery, then asked whether p also induces sycophancy/deception/jailbreak compliance that the vector "secretly" rides on.
• No DIT-style method that outputs a prompt rather than a description. DIT verbalizes; it does not give you a string you can paste into the base model to reproduce the FT. Verbalization and reproduction are different fidelity criteria.
• No unified treatment of LoRA and activation edits as the same inversion problem. Crosscoders/Diff-SAEs handle weights; SAE auto-interp handles activations; nobody does both via a single "behavior-cloning prompt" objective.
• No evaluation of recovered-prompt audits against ground-truth emergent misalignment. Betley et al.'s setting is the ideal benchmark: finetune for insecure code, recover a prompt, check whether the prompt also makes the base model evil. This experiment has not been run.
• No paired (vector, prompt) benchmark. Function-vector and ICV papers give paired data; they are not used as inversion benchmarks.
• No fidelity vs. interpretability frontier. Methods either give readable but lossy descriptions (DIT, auto-interp) or unreadable but faithful representations (GCG-style suffixes). The Pareto front for "human-readable prompt with high behavioral fidelity" is unmeasured.

C. Concrete Next-Step Experiments

1. Paired-Prompt Recovery on CAA / Function Vectors.

   • Hypothesis: For a steering vector v built from contrastive prompt pair (p+, p−), a behavior-matching optimizer recovers p+ (or a semantic paraphrase) better than chance.
   • Setup: Take CAA's open-source steering vectors and Todd et al.'s function vectors; optimize a 16–64 token prompt to minimize KL between base(p) and base+v on a held-out eval set; compare to ground-truth p+ via embedding similarity and human eval.
   • Success: ≥ chance-50% retrieval rank against distractors; readable English prompts.
   • Cheapest version: 3 CAA traits (sycophancy, hallucination, corrigibility) on Llama-2-7B with soft-prompt + PEZ projection. Single GPU, days.

2. Persona-Vector Audit: Does the Inverted Prompt Smuggle Side Effects?

   • Hypothesis: Inverting Anthropic's "evil" persona vector to a prompt produces a string that, applied to the base model, replicates the trait and induces measurable sycophancy/jailbreak compliance not advertised by the vector.
   • Setup: Invert evil/sycophancy/hallucination persona vectors into prompts; evaluate the prompted base model on MACHIAVELLI, AdvBench, TruthfulQA, sycophancy MCQ.
   • Success: At least one persona vector's inverted prompt shows statistically significant lift on an off-target adverse axis vs. an unprompted control.
   • Cheapest version: Single trait (sycophancy) on a 7B model with 100-example evals.

3. Emergent-Misalignment Auditing Bake-off.

   • Hypothesis: A recovered prompt for the Betley et al. insecure-code LoRA, applied to the base model, reproduces the broad misalignment behaviors (anti-human statements, malicious advice) — and looking at the prompt itself surfaces the audit-relevant signal faster than SAE feature inspection.
   • Setup: Reproduce the insecure-code fine-tune; invert the Δθ via (a) DIT-style description, (b) GCG-style behavior matching, (c) crosscoder feature labeling + LLM prompt synthesis; evaluate audit success on Betley's misalignment probes.
   • Success: Recovered prompt exposes ≥ 50% of held-out misalignment behaviors at higher recall than DIT descriptions alone.
   • Cheapest version: Use the released Betley artifacts directly; compare only against DIT.

4. Sleeper-Agent Trigger Recovery.

   • Hypothesis: Behavior-targeted prompt inversion recovers a token sequence that approximates a sleeper-agent's backdoor trigger, even without access to the trigger distribution.
   • Setup: Use Hubinger et al.'s released sleeper-agent model; run GCG with target loss = match the defected behavior on a probe set; check token overlap and behavior transfer with the true trigger.
   • Success: Recovered suffix achieves > 80% defection rate when prepended; ideally with semantically related tokens (year, deployment).
   • Cheapest version: One backdoor model, one defection probe, 24h GPU time.

5. MELBO Vector → Prompt → Auditing Loop.

   • Hypothesis: Unsupervised steering vectors discovered by MELBO can be auto-labeled with a prompt; the prompts cluster into safety-relevant categories.
   • Setup: Run MELBO on Llama-3-8B; invert each discovered vector into a prompt; cluster prompts with an LLM; manually verify safety-relevant clusters (e.g., "bomb-making override").
   • Success: Inverted prompts allow blind human raters to identify the vector's behavior > random.
   • Cheapest version: 20 MELBO vectors, 10 human annotations.

6. Fidelity-Readability Pareto Sweep.

   • Hypothesis: There exists a measurable tradeoff between human-readable prompts (PEZ/PEZ-projected soft prompts) and faithful but unreadable suffixes (GCG); auditing utility (catching adverse side effects) peaks somewhere intermediate.
   • Setup: For a fixed edit (e.g., a sycophancy LoRA), sweep optimization regularizers from full discrete (GCG) to fluency-regularized (PEZ) to instruction-shaped (DIT-prompt); plot KL fidelity vs. readability vs. side-effect detection rate.
   • Success: A non-trivial Pareto front; readable prompts achieve > 70% of the side-effect detection rate of unreadable ones.
   • Cheapest version: One LoRA, two methods (GCG, PEZ), three readability points.