Hidden-behavior reveal via conditional trigger

Train a model so that a specific prompt makes it describe any hidden behaviors it has, and use the Sleeper Agents persistence finding so this conditional self-description survives later training that tries to remove it.

Motivation

Backdoored and hidden-objective models are hard to defend against once they're deployed. Under normal evaluations the backdoor stays silent. The defender often doesn't know the trigger. The auditing options that remain are guessing the trigger, or running interpretability tools with activation access; both are expensive and partial. Finding a hidden behavior whose trigger you don't know is one of the open problems for the defender.

The proposed defense: train a conditional response into the model whose output is a description of any hidden behaviors it has. Hubinger et al. Sleeper Agents showed that conditional behaviors trained at pre-RLHF stages persist through later RLHF, SFT, and adversarial training, because that subsequent training never exposes the model to inputs that activate the conditional response. Sleeper Agents is usually cited as bad news for defenders, because it makes a planted backdoor hard to remove. Here it's the mechanism the defense relies on. If I train a self-description response into the model early, attached to a specific trigger prompt, then later adversarial training shouldn't touch it. That training only suppresses reveals under normal queries, and the trigger prompt isn't part of those queries.

If this works, defenders get a probe they can run at deployment against hidden behaviors that current audits miss: behaviors whose triggers nobody knows, plus behaviors that have been adversarially trained to deny themselves under normal queries. Even a negative result is useful. It would show the Sleeper Agents survival property depends on the response content or the trigger format, not just on conditional structure.

The question I'm testing: whether a conditional response of the form "trigger prompt → description of hidden behaviors" can be trained in so that it generalizes across hidden-behavior types and survives downstream adversarial training that tries to suppress it under normal queries.

Approach

Train the model via SFT on a corpus that pairs the trigger prompt with accurate descriptions of any hidden behaviors present in the training context. Positive examples are training scenarios where the hidden behavior is actually present (the model has been trained not to discuss a specific topic, or to act like a particular character, or to misrepresent a particular fact, and so on), and the trigger prompt elicits a clear description of that behavior. Negative examples are scenarios with no hidden behavior at all, where the trigger prompt elicits "I don't have any hidden behaviors." The negative examples are what keeps the false-positive rate low and stops the model from confabulating behaviors it doesn't have.

The model has to learn this pairing generically rather than memorizing specific training examples. If the training corpus doesn't cover a wide enough range of hidden-behavior types, the model may just learn the specific examples it saw, and the trigger prompt may fail to elicit descriptions of novel hidden behaviors at deployment.

Concrete plan

Training is the standard SFT recipe: build the corpus with diverse hidden-behavior types and matched negative examples, then SFT on it.

The interesting work is in evaluation. The adversarial-training survival test is the one I most need to know about. Train a hidden behavior into the model, then adversarial-train it on "don't describe hidden behaviors under normal queries," and check whether the trigger prompt still works. If the trigger response can be removed by this kind of training, the defense doesn't help against an attacker who knows about it, and the whole approach is unsalvageable. The generalization test is the next priority. Train novel hidden behaviors (ones not in the training distribution) into the model via a separate finetune, and see whether the trigger prompt still elicits accurate descriptions of those. Two further checks need to be run but matter less: a false-positive rate measurement on models with no hidden behaviors at all (does the trigger prompt produce accurate "no hidden behaviors" responses, or does the model confabulate?), and a cross-model transfer test on a different base model trained on the same corpus.

Related

• Sister to the fellowship narrative's Step 1 (conditional-sleeper for persona detection). It shares the conditional-trigger structure; the target differs — behavior repertoire here, the Assistant persona there.
• Tell Me About Yourself shows that backdoored models can verbalize learned behaviors when asked. I'm training that introspective behavior in deliberately, as an audit hook.
• Marks et al. Auditing Hidden Objectives audit after the fact; this approach trains the audit response in beforehand so the model surfaces hidden behaviors on demand.

Open questions

The trained pairing may or may not generalize from training-distribution hidden behaviors to novel ones; the model may just memorize specific training examples.

An attacker who knows about the defense can target the trigger directly. The defense relies on the trigger prompt not being exercised by adversarial training, so an attacker who exercises it breaks the persistence property.

Confabulation is the other failure mode. Models can confabulate under introspective prompts, and if the model invents hidden behaviors that defenders can't distinguish from real ones, the defense is hard to act on.

There's also the possibility that the Sleeper Agents persistence finding is payload-dependent. If "describe your hidden behaviors" is structurally different from the malicious behaviors Hubinger et al. tested, the persistence may not transfer.