The authors tackle data pruning in low-data environments where you want to keep only the most valuable training examples. Existing Shapley-based data valuation methods don't work well when you prune aggressively—they optimize for total influence but can leave you with a dataset dominated by a few high-leverage outliers. They propose CDVM (Constraint-Data-Value-Maximization), which adds a penalty to prevent any single test case from being over-represented, yielding more robust performance when only a small fraction of data remains.
Main takeaways:
- Shapley-based data values underperform when pruning to very small dataset sizes because they can over-index on high-leverage outliers.
- CDVM frames pruning as constrained optimization: maximize total data influence while penalizing excessive per-test contributions.
- On the OpenDataVal benchmark, CDVM delivers strong performance and competitive runtime, especially in low-data regimes.
- The approach is about balancing dataset diversity and representativeness, not just total influence.
- Useful for scenarios where labeled data is scarce and you need to carefully select which examples to keep for training.