HoReN is a model-editing method that stores factual updates in an external codebook attached to a single MLP layer, leaving base weights untouched. Each codebook entry is both a memory key and a Hopfield "stored pattern," and queries/keys are normalized onto the unit hypersphere so retrieval uses angular similarity, fixing magnitude mismatches between an edit prompt and its paraphrases. The query is then refined via damped Hopfield dynamics, pulling paraphrases into the correct basin of attraction while leaving unrelated queries alone. HoReN scales to 50K sequential edits on ZsRE with stable performance above 0.9, while prior editors degrade before 10K.
Main takeaways:
- HoReN attaches a discrete key-value codebook to one MLP layer; each entry is a Hopfield stored pattern, enabling associative retrieval.
- Keys and queries are projected onto the unit hypersphere, so retrieval is governed by angular similarity, removing magnitude-driven mismatches.
- Queries are refined via damped Hopfield attractor dynamics, so paraphrases relax into the correct pattern's basin while unrelated inputs stay undisturbed.
- HoReN scales to 50K sequential edits with stable overall performance >0.9, far outperforming prior editors that collapse before 10K.