The ongoing advancements in network architecture design have led to remarkable achievements in deep learning across various challenging computer vision tasks. Meanwhile, the development of neural architecture search (NAS) has provided promising approaches to automating the design of network architectures for lower prediction error. Recently, the emerging application scenarios of deep learning have raised higher demands for network architectures considering multiple design criteria: number of parameters/floating-point operations, and inference latency, among others. From an optimization point of view, the NAS tasks involving multiple design criteria are intrinsically multiobjective optimization problems; hence, it is reasonable to adopt evolutionary multiobjective optimization (EMO) algorithms for tackling them. Nonetheless, there is still a clear gap confining the related research along this pathway: on the one hand, there is a lack of a general problem formulation of NAS tasks from an optimization point of view; on the other hand, there are challenges in conducting benchmark assessments of EMO algorithms on NAS tasks. To bridge the gap: (i) we formulate NAS tasks into general multi-objective optimization problems and analyze the complex characteristics from an optimization point of view; (ii) we present an end-to-end pipeline, dubbed $\texttt{EvoXBench}$, to generate benchmark test problems for EMO algorithms to run efficiently -- without the requirement of GPUs or Pytorch/Tensorflow; (iii) we instantiate two test suites comprehensively covering two datasets, seven search spaces, and three hardware devices, involving up to eight objectives. Based on the above, we validate the proposed test suites using six representative EMO algorithms and provide some empirical analyses. The code of $\texttt{EvoXBench}$ is available from $\href{https://github.com/EMI-Group/EvoXBench}{\rm{here}}$.

翻译：网络架构设计不断进步 { 网络架构设计不断进步 { 导致在深入学习各种具有挑战性的计算机愿景任务方面取得显著成就 。 与此同时, 神经架构搜索(NAS)的发展为将网络架构设计自动化以降低预测错误提供了有希望的方法。 最近, 正在形成的深层次学习应用情景提高了对网络架构的需求, 考虑多种设计标准: 参数/ 浮点操作的数量, 以及推导时间等。 从优化角度看, 涉及多重设计标准的NAS任务本质上是多重目标优化问题; 因此, 采取进化性多目标优化算法(EMO) 来解决这些问题是合理的。 尽管如此, 仍然存在着清晰的差距, 连接着这一路径的相关研究: 一方面, 缺乏从优化角度出发的对网络架构设计的一般性问题; 另一方面, 在对 NAS 任务中的 EMO 算法进行基准评估方面存在挑战。 (i) 我们将NAS 任务发展成一般的多目的优化问题, 并从两个优化点分析复杂特征; (ii) 我们提出运行了EVTO 测试基底 3 测试基 。