The authors decompose an evolutionary mixture-of-LoRA system into three components—a routing rewrite, a per-domain evaluation scope, and an adapter lifecycle (death, inheritance, mutation, reallocation)—and run ablation experiments to measure each piece's contribution. On their custom 150M-parameter substrate, the router rewrite carries the entire improvement (+0.043 nats), the evaluation scope does nothing, and the lifecycle actually hurts performance (-0.028 nats). The headline full-system improvement is small (+0.015 nats) and doesn't reach statistical significance, and evolutionary search only helps when adapters are already aligned to the task.
Main takeaways:
- The routing mechanism (how adapter outputs are combined) drives all the measured improvement in this evolutionary LoRA setup
- The lifecycle component (adapter mutation and reallocation) is a net drag on performance in these experiments
- Evolutionary search on routing is only useful when adapters are pre-aligned to tasks; otherwise it doesn't help
- Statistical power is limited (n=3 seeds) and the headline result doesn't clear significance thresholds
- There's a substrate-conditional regime boundary: what works depends heavily on initialization and model architecture