This work proposes a Processing-In-Sensor Accelerator, namely PISA, as a flexible, energy-efficient, and high-performance solution for real-time and smart image processing in AI devices. PISA intrinsically implements a coarse-grained convolution operation in Binarized-Weight Neural Networks (BWNNs) leveraging a novel compute-pixel with non-volatile weight storage at the sensor side. This remarkably reduces the power consumption of data conversion and transmission to an off-chip processor. The design is completed with a bit-wise near-sensor processing-in-DRAM computing unit to process the remaining network layers. Once the object is detected, PISA switches to typical sensing mode to capture the image for a fine-grained convolution using only the near-sensor processing unit. Our circuit-to-application co-simulation results on a BWNN acceleration demonstrate acceptable accuracy on various image datasets in coarse-grained evaluation compared to baseline BWNN models, while PISA achieves a frame rate of 1000 and efficiency of ~1.74 TOp/s/W. Lastly, PISA substantially reduces data conversion and transmission energy by ~84% compared to a baseline CPU-sensor design.

翻译：这项工作提议了一个处理传感器加速器, 即 PISA, 作为一种灵活、 节能和高性能的解决方案, 用于实时和智能图像处理。 PISA 内在地在Binizizized- Weight神经网络(BWNNS)中执行粗微重感应共变操作, 利用传感器侧面非挥发性重量储存的新型计算像素。 这明显减少了数据转换和传输到离芯处理器的能量消耗。 设计完成时, 使用一个略微近传感器处理DRAM 计算器处理剩余网络层。 一旦发现该对象, PISA 开关到典型的感测模式, 仅使用近传感器处理器来捕捉微微重心变动图像。 BWNNN加速的电路到应用共模拟结果显示, 与基线模型相比, 各种图像数据集的精确度为可接受。 PISA 达到1000 框架率, 并且通过 ~PIA/ TONNNM 和最终的能量转换率 。