The authors built PIVOT, a system that helps LLM agents fix their own plans by executing them, seeing what goes wrong, and iteratively refining. The key idea is treating entire trajectories (sequences of actions) as objects you can optimize: the system generates a plan, runs it to collect structured error signals ("textual gradients"), uses those to evolve better plans, and verifies the final result against constraints. On planning benchmarks, PIVOT hit 94% relative improvement in constraint satisfaction with human feedback and retained substantial gains fully autonomously, while using 3–5× fewer tokens than competing methods.
Main takeaways:
- Plans are refined through a four-stage loop: generate candidate trajectories, execute and collect error feedback, evolve improved versions, verify constraints.
- A monotonic acceptance rule ensures solution quality never decreases across iterations.
- Human-in-the-loop feedback gives the biggest gains, but the core self-supervised loop (no human help) still improves substantially over static planning.
- Much more token-efficient than other refinement approaches—uses a fraction of the compute for similar or better results.
- Shows that feedback-driven trajectory optimization is a principled way to close the gap between what an agent plans and what actually works in execution.