Diffusion models for image generation are biased because of discretization and imperfect score function estimates. Existing corrector steps (like unadjusted Langevin) don't fully fix this. The authors propose using Metropolis-Hastings or Barker accept-reject steps to eliminate the bias from discretization. Since the usual target density ratio isn't available, they show how to compute correct acceptance probabilities using the score function instead. They introduce the first exact correction via a two-coin Bernoulli factory and a practical approximation using Simpson's rule that's very accurate and nearly free computationally. Experiments show improved sample quality (better FID scores) on image datasets.
Main takeaways:
- Standard corrector steps in diffusion models (like unadjusted Langevin) are themselves biased due to discretization
- Metropolis-Hastings or Barker corrections can remove this bias, but require a target density ratio that's unavailable
- New methods compute acceptance probabilities using only the score function
- An exact correction uses a Bernoulli factory; a practical approximation uses Simpson's rule with order 5/2 accuracy
- Empirical results show consistent improvements in image quality (FID) on synthetic and real datasets