The authors develop a method to improve Bayesian posterior sampling by combining variational inference (VI) with a resampling trick. Standard VI methods like mean-field approximations are fast but often produce overly confident, under-dispersed posteriors that miss correlations between parameters. Their variational predictive resampling (VPR) approach works by repeatedly generating fake future data from the current variational approximation, updating the approximation given this synthetic data, and recording the implied parameter values — essentially building better posterior samples from VI's predictive strength without expensive MCMC.
Main takeaways:
- Mean-field VI is computationally cheap but often yields overly narrow posteriors that miss parameter dependencies
- VPR repeatedly imputes synthetic observations from the predictive distribution, updates the variational approximation, and collects the resulting parameters
- In a tractable Gaussian example, VPR recovers the exact Bayesian posterior while standard mean-field VI retains a permanent error
- Experiments on regression and hierarchical models show VPR captures posterior uncertainty and correlations that mean-field misses, at computational cost similar to or better than MCMC