CLIPer uses a lightweight classifier to steer LLM generation at inference time toward different user preferences (helpfulness, conciseness, humor, etc.) without fine-tuning a separate model for every preference combination. The classifier guides generation dynamically, adding negligible computational cost while enabling controllable personalization across single and multi-dimensional preferences. Empirical results show the approach scales well and delivers effective personalized generation without extensive training.
Main takeaways:
- Eliminates the need to fine-tune separate models for each preference combination (helpfulness+concise, helpful+humorous, etc.)
- Uses a classifier model to dynamically steer generation toward desired preferences at inference time
- Works across single preferences (just conciseness) and multi-dimensional combinations (concise + helpful)
- Adds negligible computational overhead compared to fine-tuning multiple models
- Enables more nuanced control over generation style without retraining