Artificial intelligence (AI) research today is largely driven by ever-larger neural network models trained on graphics processing units (GPUs). This paradigm has yielded remarkable progress, but it also risks entrenching a hardware lottery in which algorithmic choices succeed primarily because they align with current hardware, rather than because they are inherently superior. In particular, the dominance of Transformer architectures running on GPU clusters has led to an arms race of scaling up models, resulting in exorbitant computational costs and energy usage. At the same time, today's AI models often remain unreliable in the sense that they cannot properly quantify uncertainty in their decisions -- for example, large language models tend to hallucinate incorrect outputs with high confidence. This article argues that achieving more efficient and reliable AI will require embracing a set of principles that are well-aligned with the goals of neuromorphic engineering, which are in turn inspired by how the brain processes information. Specifically, we outline six key neuromorphic principles, spanning algorithms, architectures, and hardware, that can inform the design of future AI systems: (i) the use of stateful, recurrent models; (ii) extreme dynamic sparsity, possibly down to spike-based processing; (iii) backpropagation-free on-device learning and fine-tuning; (iv) probabilistic decision-making; (v) in-memory computing; and (vi) hardware-software co-design via stochastic computing. We discuss each of these principles in turn, surveying relevant prior work and pointing to directions for research.
翻译:当前的人工智能研究主要由基于图形处理器训练的大型神经网络模型驱动。这一范式取得了显著进展,但也可能固化为一种硬件彩票效应,即算法的成功主要源于其与当前硬件特性的契合,而非其内在优越性。特别是,在GPU集群上运行的Transformer架构主导地位引发了模型规模竞赛,导致高昂的计算成本和能源消耗。同时,当前AI模型在可靠性方面仍存在不足,无法准确量化决策中的不确定性——例如,大语言模型常以高置信度产生错误的幻觉输出。本文认为,要实现更高效可靠的人工智能,需要采纳一系列与神经形态工程目标相契合的原理,这些原理源自大脑信息处理机制的启发。具体而言,我们提出六项涵盖算法、架构与硬件的关键神经形态原理,可为未来AI系统设计提供指导:(i)采用有状态的循环模型;(ii)极端的动态稀疏性,可能延伸至脉冲式处理;(iii)无需反向传播的端侧学习与微调;(iv)概率化决策机制;(v)存内计算技术;(vi)通过随机计算实现软硬件协同设计。我们将依次探讨这些原理,综述相关前沿研究并指出未来研究方向。