CoCoDA organizes tool-augmented LLM agents using a compositional code DAG (directed acyclic graph) where nodes are primitive or composite tools and edges show invocation dependencies. At inference, it retrieves tools by progressively filtering: first by type signatures, then descriptions, then behavioral specs, then examples—keeping expensive context materialization for only the smallest candidate sets. During training, successful action sequences become new composite tools, and the planner gets credit proportional to how many primitives each composite saves. Results show an 8B model matching a 32B teacher on math benchmarks.
Main takeaways:
- Organizes tools in a typed DAG rather than flat text-indexed memory, exploiting compositional structure
- Retrieval uses a funnel: type-check → semantic rank → spec filter → example disambiguate, progressively shrinking candidates
- Successful trajectories are "folded" into new composite tools; planner reward credits composites by primitive count saved
- Theoretical guarantees: sublinear retrieval cost, monotone improvement under conservative updates, DAG stays well-formed
- 8B student matches 32B teacher on GSM8K and MATH; consistent gains across benchmarks