Computer-use agents (CUAs) that control browsers, terminals, and applications face reliability challenges beyond task success: perception errors, planning drift, permission scope, and runtime oversight all affect whether actions stay aligned with user intent. This paper develops a framework organized by architectural layers (Perception → Decision → Execution) and lifecycle stages (Creation → Deployment → Operation → Maintenance) to analyze where failures are introduced versus where they become visible, and maps intervention surfaces for control and assurance.
Main takeaways:
- Agent reliability isn't just task success—it includes perception accuracy, planning stability, memory use, tool safety, and permission boundaries
- The framework separates architectural layers (how agents transform observations into actions) from lifecycle stages (when priors are learned, tools bound, and drift occurs)
- Failures often become visible in different stages than where their root causes are introduced
- Identifies open challenges: controllable grounding, long-horizon constraint preservation, safe authority binding, runtime defense