Skip to content
Sagan

Paper

UCCI: Calibrated Uncertainty for Cost-Optimal LLM Cascade Routing

Unreadunread

AI summary

arXiv:2605. 18796v1 Announce Type: new Abstract: LLM cascades and model routing promise lower inference cost by sending easy queries to a small model and escalating hard ones to a large model, but most deployed routers use uncalibrated confidence scores and require per-workload threshold tuning.