In visual interactive labeling, users iteratively assign labels to data items until the machine model reaches an acceptable accuracy. A crucial step of this process is to inspect the model's accuracy and decide whether it is necessary to label additional elements. In scenarios with no or very little labeled data, visual inspection of the predictions is required. Similarity-preserving scatterplots created through a dimensionality reduction algorithm are a common visualization that is used in these cases. Previous studies investigated the effects of layout and image complexity on tasks like labeling. However, model evaluation has not been studied systematically. We present the results of an experiment studying the influence of image complexity and visual grouping of images on model accuracy estimation. We found that users outperform traditional automated approaches when estimating a model's accuracy. Furthermore, while the complexity of images impacts the overall performance, the layout of the items in the plot has little to no effect on estimations.

翻译：在视觉互动标签中,用户在机器模型达到可接受的准确度之前迭接地指定数据项标签,这一过程的关键步骤是检查模型的准确性,并决定是否有必要标出其他元素。在没有或很少贴标签的数据的情况下,需要对预测进行视觉检查。通过维度减缩算法产生的相似性-保留散射图是这些情况中使用的一种常见的可视化。以前的研究调查了布局和图像复杂性对标签等任务的影响。然而,模型评估尚未系统研究。我们介绍了一项实验的结果,研究图像复杂性和图像的视觉组合对模型准确性估计的影响。我们发现,在估计模型准确性时,用户比传统的自动化方法要强。此外,虽然图像的复杂性影响到整体性能,但图中项目的布局对估计几乎没有任何影响。