The authors analyzed LLM reasoning by extracting explicit search trees from chain-of-thought (CoT) traces in a four-in-a-row board game, comparing them to human planning. They found that while LLMs write about deep lookahead (many moves into the future), their actual move choices are best explained by a myopic model that only considers shallow nodes—LLMs don't act on the deep reasoning they generate, unlike humans whose performance is driven by deeper search.
Main takeaways:
- LLMs' search trees are shallower than humans', and their performance correlates with search breadth (exploring many options) rather than depth (looking many moves ahead).
- Even though LLMs generate reasoning traces that mention deep nodes (many steps into the future), their move decisions are best predicted by ignoring those deep nodes entirely.
- A causal intervention—selectively removing shallow vs. deep paragraphs from CoT—confirmed that shallow nodes drive move selection, not deep ones.
- This contrasts with human planning, where expertise comes from deeper search (thinking further ahead).
- Suggests a fundamental gap: LLMs generate the appearance of deep planning but don't use it for decisions, offering targeted guidance for alignment.