Computer Use Agents (CUAs) are designed to autonomously operate digital interfaces, yet they often fail to reliably determine whether a given task has been completed. We present an autonomous evaluation and feedback framework that uses vision-language models to assess task completion directly from screenshots and task descriptions. Our dataset covers 42 built-in macOS applications and 1,260 human-labeled tasks across a wide range of scenarios. Our framework achieves up to 73 percent accuracy in task success detection and yields an average relative improvement of 27 percent in overall task success when evaluator feedback is applied. These results show that vision-based evaluation can serve as an effective feedback mechanism that improves the reliability and self-correction of autonomous computer-use agents.

翻译：计算机使用代理旨在自主操作数字界面，但其往往难以可靠判断给定任务是否已完成。本文提出一种自主评估与反馈框架，利用视觉语言模型直接从屏幕截图和任务描述中评估任务完成度。我们的数据集涵盖42个内置macOS应用程序及1,260个人工标注任务，覆盖广泛场景。该框架在任务成功检测中达到最高73%的准确率，应用评估反馈后整体任务成功率平均相对提升27%。结果表明，基于视觉的评估可作为有效反馈机制，提升自主计算机使用代理的可靠性与自我修正能力。