The Bicameral Model connects two frozen pretrained language models through a tiny learned interface (~1% of total parameters) that lets them exchange information via hidden states rather than text. One model handles the main task while the other operates tools (calculator, logic solver, code interpreter), and both condition on each other's internal activations at every generation step. A learned "suppression gate" decides what gets communicated, trained only from task loss without any prescribed protocol. On arithmetic, coupling two 0.5B models with a calculator jumps accuracy from 36% to 96%.
Main takeaways:
- Two frozen LMs can coordinate through continuous hidden-state channels instead of generating text back and forth
- A trainable translation network plus suppression gate (~1% parameters) learns a communication protocol from task loss alone
- On arithmetic (calculator backend), accuracy rises from 36% to 96% with two 0.5B models
- On logic puzzles (Z3 solver backend), 1.7× improvement over baseline with two 0.6B models
- The auxiliary model can generate problem-specific code from hidden-state signals without ever seeing the problem text