The massive amounts of data generated by camera sensors motivate data processing inside pixel arrays, i.e., at the extreme-edge. Several critical developments have fueled recent interest in the processing-in-pixel-in-memory paradigm for a wide range of visual machine intelligence tasks, including (1) advances in 3D integration technology to enable complex processing inside each pixel in a 3D integrated manner while maintaining pixel density, (2) analog processing circuit techniques for massively parallel low-energy in-pixel computations, and (3) algorithmic techniques to mitigate non-idealities associated with analog processing through hardware-aware training schemes. This article presents a comprehensive technology-circuit-algorithm landscape that connects technology capabilities, circuit design strategies, and algorithmic optimizations to power, performance, area, bandwidth reduction, and application-level accuracy metrics. We present our results using a comprehensive co-design framework incorporating hardware and algorithmic optimizations for various complex real-life visual intelligence tasks mapped onto our P2M paradigm.

翻译：摄像头传感器生成的海量数据促使对像素阵列内部、即极限边缘处的数据进行处理。最近几年关于像素内部处理-in-memory的兴趣快速增长，包括以下几个方面的关键发展：(1)三维集成技术的进步，使得每个像素内部的复杂处理在保持像素密度的同时也变得可能；(2)用于大规模并行低功耗像素内部计算的模拟电路技术；(3)算法技术，通过硬件感知式训练方案来减轻与模拟处理相关的非理想效应。 本文提出了一个综合技术-电路-算法的系统，该系统将技术能力、电路设计策略和算法优化与功耗、性能、面积、带宽降低和应用级的精度指标相结合。 我们利用一个全面的协同设计框架，结合硬件和算法优化，将各种复杂的现实视觉智能任务与我们的P2M范式相映射，并展示了相关实验结果。