The authors study "political plasticity"—how easily LLMs adapt their political stance based on context—using 200 questions about economic and personal freedom. They find that system prompts barely work, but user prompts with few-shot examples successfully shift models' positions, especially on economic issues in larger/newer models. Flipping question polarity revealed counter-intuitive responses suggesting data leakage, and plasticity varied subtly across languages.
Main takeaways:
- System prompts were largely ineffective at inducing political bias; user prompts with examples worked much better
- Larger and newer models showed stronger ideological adaptability, particularly on economic freedom questions
- Inverting question sense caused unexpected shifts, hinting at possible questionnaire format memorization (data leakage)
- Small/old models showed limited or unstable plasticity; frontier models were reliably adaptable
- Language choice produced subtle but noticeable differences in political stance