The paper analyzes how data augmentation regularizes regression models when the number of features grows proportionally with the number of samples (the "proportional regime"). They derive a tight formula for test error (mean squared error) that depends only on the true data distribution and simple statistics of the augmentation scheme—first and second moments. The results apply even when the feature map is misspecified and hold for any network where only the final layer is trained (frozen or random features below).
Main takeaways:
- Characterizes test error in the proportional regime (features scale with samples) in terms of population quantities and augmentation statistics
- Works under model misspecification and for any architecture with a frozen feature extractor and trainable readout layer
- Provides concrete results for Gaussian data showing the asymptotic formulas are tight
- Explains the regularization effect of data augmentation through first and second order statistics of the augmentation distribution