The authors tackle gene regulatory network (GRN) inference—figuring out which genes regulate which other genes—using single-cell foundation models (scFMs). They find that standard scFMs underperform because their pretraining (reconstruction-based objectives) doesn't explicitly learn regulatory signals. They introduce two new methods, Virtual Value Perturbation and Gradient Trajectory, to extract implicit regulatory knowledge from scFMs and build a zero-shot benchmark that tests whether models can predict regulation for genes and datasets they've never seen.
Main takeaways:
- Single-cell foundation models don't naturally capture gene regulation—their reconstruction pretraining misses latent regulatory signals.
- The new benchmark tests zero-shot generalization: can a model predict regulatory relationships on completely unseen genes and datasets?
- Virtual Value Perturbation and Gradient Trajectory distill regulatory knowledge from scFMs into transferable inter-gene features.
- The approach significantly outperforms existing GRN inference methods, especially on out-of-distribution data.
- This creates a new paradigm for using foundation models in biology: extracting implicit structure rather than using embeddings directly.