The authors test whether LLMs learn in-context by pattern-matching recent tokens or by inferring latent structure, using a task where models do random walks on graphs with competing topologies. Using PCA on internal representations, they find both graph structures are encoded in orthogonal subspaces simultaneously, suggesting genuine structure learning. Activation patching and steering experiments causally confirm that late layers encode graph-family preferences that can be transferred or manipulated, pointing to a dual mechanism combining structure inference and local copying.
Main takeaways:
- PCA reveals that models encode multiple graph topologies in orthogonal subspaces, not just local transition patterns
- Late-layer activation patching almost fully transfers graph preference from one context to another
- Linear steering successfully moves predictions toward different graph families; control conditions (norm-matched, label-shuffled) fail
- Evidence supports dual mechanisms: both genuine structure inference and local pattern-matching operate in parallel
- The graph task provides a clean setting where structure vs. pattern-matching is in principle decidable