SkillGen is a system that automatically extracts reusable "skills" (auditable procedures) from an LLM agent's successful and failed trajectories. Rather than just summarizing what worked, it contrasts successes against failures to identify reliable patterns, common failure modes, and missing steps. It then generates candidate skill descriptions, iteratively refines them, and verifies each skill by running the same instances with and without the skill to measure net improvement (accounting for both fixes and regressions). Experiments show SkillGen consistently improves held-out performance, beats summarization baselines, and produces skills that transfer across models.
Main takeaways:
- Most agent skills are still hand-written; SkillGen synthesizes them automatically from execution traces.
- The system uses contrastive induction: comparing successful and failed trajectories to find reusable success patterns and recurring failure modes.
- Skills are human-readable artifacts that can be inspected before deployment, not opaque learned weights.
- A key novelty is modeling skills as interventions: SkillGen tests the same instances with and without the skill to measure repairs (fixed failures) minus regressions (broken successes).
- Across multiple agents and datasets, SkillGen outperforms existing skill-generation baselines and produces skills that transfer to different base models.