The authors built a comprehensive benchmark (ASD-Bench) to evaluate how well different machine learning models can screen for autism spectrum disorder using questionnaire data. They tested everything from classic ML (XGBoost, logistic regression) to modern deep learning and transformer models across three age groups (children, adolescents, adults), measuring not just accuracy but also calibration, interpretability, and robustness. The benchmark reveals that adult classification is easy (many models get perfect scores) but adolescent screening is much harder, and that the most important questionnaire features shift dramatically by age—social motivation matters most for children, pattern recognition for adolescents, and adults show a flatter profile consistent with social masking.
Main takeaways:
- Adult ASD screening from questionnaires is nearly solved (10 of 17 models hit perfect F1), but adolescent classification is significantly harder (F1 ceiling of 0.837 vs 0.915 for children)
- Feature importance shifts by developmental stage: social motivation (question A9) dominates for children, pattern recognition (A5) for adolescents, adults show flatter profiles
- High accuracy doesn't guarantee good calibration—one model achieved perfect F1 but had terrible calibration (ECE=0.302), showing you can't rely on a single metric for clinical AI
- They introduced HAP (Heuristic Aggregate Penalty), a metric that penalizes false negatives more heavily and accounts for cross-validation stability, which is more appropriate for medical screening than standard metrics