The authors built SCALAR, an Actor–Critic–Judge pipeline for theoretical physics problems, to study how multi-turn dialogue between a researcher-like Actor and a Critic affects solution quality. They found that multi-turn always beats single-shot attempts, but the best feedback strategy (constructive, lenient, strict, or adversarial) depends heavily on the Actor–Critic pairing—constructive feedback helps most when a lightweight Actor is guided by a stronger Critic, while same-family pairings show weaker strategy effects and sometimes favor lenient feedback.
Main takeaways:
- Multi-turn dialogue consistently improves over single-shot across all Actor–Critic pairings tested on quantum field theory and string theory problems.
- Feedback strategy matters most in asymmetric pairings (weak Actor + strong Critic), where constructive feedback improves mean scores.
- In same-family pairings (e.g., DeepSeek-R1 8B Actor + DeepSeek-R1 8B Critic), strategy effects are weaker; lenient feedback sometimes helps, strict and adversarial don't.
- Scaling within one family (8B to 70B DeepSeek-R1) improves easier problems but doesn't remove the hardest bottleneck.
- The mechanism of improvement and value of prompting choices depend strongly on the specific Actor–Critic combination, not uniformly across all setups.