Many experts argue that artificial superintelligence (ASI), if developed prematurely, poses catastrophic risks including human extinction from misaligned ASI, geopolitical instability, and misuse by malicious actors. This report proposes an international agreement to prevent the premature development of ASI until AI development can proceed without these risks. The agreement halts dangerous AI capabilities advancement while preserving access to current, beneficial AI applications. The proposed framework centers on a coalition led by the United States and China that would restrict the scale of AI training and dangerous AI research. Due to the lack of trust between relevant parties, verification is a key part of the agreement. Limits on the scale of AI training are operationalized by FLOP thresholds and verified through the tracking of AI chips and verification of chip use. Dangerous AI research--that which advances toward artificial superintelligence or endangers the agreement's verifiability--is stopped via legal prohibitions and multifaceted verification. We believe the proposal would be technically sufficient if implemented today, but advancements in AI capabilities or development methods would hurt its efficacy. Simultaneously, there is not yet political will for such an agreement. Despite these concerns, we hope this agreement can provide direction for AI governance research and policy.
翻译:许多专家认为,人工超智能(ASI)若过早发展,将带来灾难性风险,包括因未对齐的ASI导致人类灭绝、地缘政治不稳定以及恶意行为者的滥用。本报告提出一项国际协议,旨在防止在人工智能发展能够规避这些风险之前过早开发ASI。该协议在保留现有有益AI应用的同时,暂停危险AI能力的推进。所提出的框架以美国和中国的联盟为核心,限制AI训练的规模及危险的AI研究。由于相关方之间缺乏信任,核查成为协议的关键部分。对AI训练规模的限制通过浮点运算阈值实施,并通过追踪AI芯片及核查芯片使用情况来验证。危险的AI研究——即那些推动人工超智能发展或危及协议可核查性的研究——将通过法律禁令和多层面核查予以禁止。我们认为,若当前实施,该提案在技术上具有充分性,但AI能力或开发方法的进步将削弱其效力。同时,目前尚未形成对此类协议的政治意愿。尽管存在这些担忧,我们希望该协议能为AI治理研究和政策提供方向。