The paper evaluates how differential privacy training (DP-SGD) affects social bias in LLMs across four different testing paradigms: sentence scoring, text completion, tabular classification, and question answering. They find that DP reduces bias in sentence scoring (likelihood-based measurements) but the improvement doesn't carry over to other tasks, and that logit-level bias doesn't always match output-level bias. Importantly, reducing memorization through DP doesn't automatically reduce unfairness.
Main takeaways:
- DP-SGD training reduces bias on sentence scoring tasks but not consistently across completion, classification, or QA tasks
- Logit-level bias measurements (what the model "thinks") can disagree with output-level bias (what it actually generates)
- Lower memorization from differential privacy doesn't guarantee lower social bias
- Multi-paradigm evaluation is essential—conclusions from one measurement approach don't generalize
- Training modifications affect bias in task-specific, not universal ways