The authors propose using neural networks to learn feature maps that define kernels for Gaussian processes (GPs), enabling fast exact GP inference at scale. They show the learned feature map can be seen as an optimal low-rank approximation to a Gram matrix from an implied reproducing kernel Hilbert space (RKHS—a function space with an inner product defined by the kernel), and prove the GP posterior is consistent. They also introduce product feature-map kernels to avoid oversmoothing. The method handles regression and classification across tabular and image data, and benchmarks show it beats existing GP methods in accuracy and speed.
Main takeaways:
- Neural feature maps define expressive kernels that enable fast, scalable exact GP inference without expensive precomputation
- The learned feature map is provably an optimal low-rank approximation to a kernel Gram matrix, with posterior consistency guarantees
- Product feature-map kernels prevent oversmoothing by combining multiple feature maps
- Outperforms prior GP methods on benchmarks across diverse data types (tabular, images)