The authors applied cognitive psychology's Wechsler Adult Intelligence Scale (WAIS) to multimodal generative AI models, revealing a profoundly uneven cognitive profile: near-ceiling performance on verbal comprehension and working memory (>98th percentile) but near-floor perceptual reasoning (<1st percentile). They developed the AIQ Benchmark to track how these profiles evolve across six model generations, finding that abstract reasoning improves much faster when presented linguistically than visually, suggesting an architectural bias toward language-based symbolic processing.
Main takeaways:
- Leading multimodal models show extreme cognitive imbalance: excellent verbal skills but severely impaired visual-perceptual reasoning.
- Abstract quantitative reasoning improves rapidly when presented as text but lags dramatically in visually analogous formats, indicating a deep architectural preference for language.
- Visual-perceptual organization (interpreting visual spatial relationships) remained largely stagnant across generations despite improvements in abstract visual reasoning.
- The AIQ Benchmark extends beyond human-normed ceilings to track superhuman and subhuman performance on the same cognitive dimensions.
- Findings suggest that scaling and optimization alone won't overcome architectural limitations preventing balanced, human-like general intelligence.