The safety of an automated vehicle hinges crucially upon the accuracy of perception and decision-making latency. Under these stringent requirements, future automated cars are usually equipped with multi-modal sensors such as cameras and LiDARs. The sensor fusion is adopted to provide a confident context of driving scenarios for better decision-making. A promising sensor fusion technique is middle fusion that combines the feature representations from intermediate layers that belong to different sensing modalities. However, achieving both the accuracy and latency efficiency is challenging for middle fusion, which is critical for driving automation applications. We present A3Fusion, a software-hardware system specialized for an adaptive, agile, and aligned fusion in driving automation. A3Fusion achieves a high efficiency for the middle fusion of multiple CNN-based modalities by proposing an adaptive multi-modal learning network architecture and a latency-aware, agile network architecture optimization algorithm that enhances semantic segmentation accuracy while taking the inference latency as a key trade-off. In addition, A3Fusion proposes a FPGA-based accelerator that captures unique data flow patterns of our middle fusion algorithm while reducing the overall compute overheads. We enable these contributions by co-designing the neural network, algorithm, and the accelerator architecture.

翻译：自动车辆的安全关键在于感知和决策延迟度的准确性。 根据这些严格的要求,未来的自动汽车通常配备多式传感器,如相机和LiDARs。感应聚合被采用,以便为更好的决策提供充满信心的驱动情景。有希望的感应聚变技术是中间融合技术,将属于不同感测模式的中间层的特征表现结合起来。然而,实现准确性和延迟效率对于中间聚变具有挑战性,而中间聚变对于驱动自动化应用程序至关重要。我们介绍了A3Fusion,一个专门用于适应、敏捷和配合自动化的软件硬件系统。A3Fusion通过提出一个适应性、灵敏度和配合性融合多种CNN模式的中间聚变效率很高,其方法是提出一个适应性多式学习网络架构和一个耐用度、敏度和灵活的网络结构优化算法,既能提高分辨率分解的准确性,又能将推断性惯性惯性拉度作为关键交换工具。此外,A3Fusion 提出一个基于适应性、敏捷性和配合自动化的软硬件系统系统系统。我们通过这些独特的数据流算算法,从而测量了高层数据流结构。