Multimodal models that compress visual reasoning into text lose fine-grained information. Recent "latent reasoning" methods try to reason in continuous hidden states instead, but the authors find these latent trajectories drift from the model's pretrained circuits, collapse into generic patterns, and get bypassed during answer generation. They propose RIS, which grounds latent reasoning tokens to spatial bounding boxes and semantic region descriptions, enforces their causal role through a progressive attention bottleneck, and uses short language tokens to bridge back to vocabulary-aligned decoding. RIS improves over baselines on vision-reasoning benchmarks and produces more interpretable latent trajectories.
Main takeaways:
- Latent visual reasoning (reasoning in hidden states rather than text) often fails because trajectories drift from pretrained circuits and get ignored during decoding
- RIS anchors latent tokens to spatial (bounding boxes) and semantic (region descriptions) evidence to keep them grounded
- A progressive attention bottleneck forces the model to actually use latent tokens rather than bypass them
- Short language "transition tokens" help bridge continuous latent states back to vocabulary-aligned text generation