梅兰妮·W·西森，美国布鲁金斯学会高级研究员

江天骄，复旦大学发展研究院副教授，博士生导师

编者按：2026年5月，中美两国元首在北京会晤期间，就人工智能问题进行了建设性交流，同意开展人工智能政府间对话。在此背景下，清华大学战略与安全研究中心（CISS）与美国布鲁金斯学会（Brookings）“中美人工智能与国际安全二轨对话”项目组成员结合前期对话交流经验，就军事领域人工智能治理问题展开进一步探讨，为两国在该领域的良好互动提供相关思考。此篇文章首发于布鲁金斯网站，现转载以飨读者。

（以下中文翻译仅供参考，英文原文附后）

两位作者分别从美方和中方的视角，探讨两国可通过哪些举措为人工智能在国际安全领域的合规适用划定边界。相关建议仅代表作者个人观点，旨在为即将举行的中美人工智能政府间对话提供参考。文章探讨了在人工智能军事应用日益广泛的背景下，中美两国如何基于人类管控核武器的既有承诺，持续推进相关工作。作者进一步分析了双方可能达成共识的潜在领域，包括确保人类对人工智能驱动的网络攻击的控制权，以及能将共识原则付诸实践的具体机制与保障措施。

中美可在军事人工智能治理方面发挥引领作用

梅兰妮·W·西森

美国和中国尚未就军事领域人工智能治理形成共同方案。然而，这并不意味着两国无意应对这样一种可能性，即人工智能的某些军事应用可能伤害平民、危及社会，并破坏全球稳定。

2024年年底，美国和中国两国总统达成共识，认为核武器使用决策应由人类而非人工智能作出。这一共识体现了双方对核战争灾难性后果的共同认识，也反映出双方有必要尽可能降低这样一种风险：核战争的发生并非源于领导人的主动选择，而是由于人工智能出现故障或被滥用。

这一共识意义重大，但它只是起点。随着各国军队将智能体式人工智能（Agentic AI）逐步融入军事行动，以及人工智能模型在复杂程度和能力方面快速发展，导致灾难性后果的新路径正在不断出现。美国总统唐纳德·特朗普和中国国家主席习近平可以在治理这些风险方面发挥引领作用，双方可进一步达成共识，即针对彼此的核指挥、控制与通信系统（NC3）或关键基础设施发动网络攻击的决定，应由人类而非人工智能作出。

人工智能驱动的网络攻击所带来的危险

各国依靠核指挥、控制与通信系统探测威胁、作出核武器使用决策，并向负责执行相关行动的部队传输安全、可靠的发射命令。无论采取何种核战略原则，都无法完全消除这些风险。无论一个国家奉行不首先使用核武器（no-first-use）、预警即发射（launch-on-warning）还是其他原则，只要上述功能出现故障，就可能导致：领导人无法正常沟通或下达指令；国家无法探测威胁或准确评估威胁；产生错误认知并增加发动首次打击的动机；或者导致核武器被错误发射或未经授权发射。

因此，对另一国的核指挥、控制与通信系统发动蓄意的人工智能驱动网络攻击，可能引发一场具有灾难性后果的军事危机。但引发军事危机的并不只有蓄意攻击这一种方式。人工智能驱动的网络作战能力还可能通过另外两种方式意外引发军事危机。

第一，即使一国的目的仅仅是预先部署进攻性网络能力，由于人为失误、人工智能模型的技术局限，或者人工智能智能体以人类未曾预料的方式行动，仍可能意外发动攻击。

第二，恶意行为者可能利用人工智能驱动的网络能力发动未经授权的攻击。在这两种情况下，遭受攻击的一方都极有可能提高警戒级别，并使其核力量进入准备状态。如果核指挥、控制与通信系统受到的干扰严重到足以影响其正常运行，各国领导人就可能不得不在极端时间压力和高度不确定性下作出是否使用核武器的决定，这可能最终导致核武器发射乃至核战争

针对关键基础设施——包括支撑社会正常运行的经济、金融、医疗、交通和公共安全系统——发动人工智能驱动的网络攻击，同样可能引发极其严重的军事危机。这类攻击可能直接造成财产毁损、当地环境污染、社会混乱，以及大范围的疾病、伤害或死亡。与针对核指挥、控制与通信系统的攻击一样，对关键基础设施的攻击既可能是蓄意发动的，也可能源于技术错误或未经授权的行动。遭受关键基础设施攻击的国家极有可能进行军事报复，从而引发一场具有不可预测升级轨迹的战争。

总统层面的领导至关重要

即将举行的美中人工智能对话为世界主要人工智能国家提供了一个机会，使其能够在治理这些极其危险的人工智能驱动进攻性网络能力方面取得实质性进展。计划中的政府间对话可以为特朗普和习近平进一步达成共识创造条件，即针对核指挥、控制与通信系统以及关键基础设施发动网络攻击的决定，应由人类而非人工智能作出。

将适用于核武器的这一原则进一步延伸，即具有战略影响的攻击应体现人类意图，并且必须由人类承担责任，并不要求华盛顿和北京在其他网络安全原则或基础设施保护问题上达成一致。这一原则仅限于一个狭窄但重要的共识：双方都有责任尽可能降低这样的风险，即具有战略影响的攻击可能因偶然、错误、具备超越人类预设约束能力的先进人工智能，或未经授权的行为者而发生。

领导人层面的声明是就可核查的协议和安全保障措施开展双边对话的必要前提。同时，这类声明也是一种重要的公开信号，表明美国和中国认识到，作为人工智能领域领先国家，两国肩负着特殊责任，并致力于采取务实措施，降低军事应用人工智能将国家置于生存性风险之中的可能性。没有其他信号能够以同等分量表明，两国政府都清楚认识到其中的利害关系——这不仅关乎各自社会，也关乎全世界。

中美应当加快“人类控制”原则落地

江天骄

人工智能（AI）正以前所未有的速度渗透到军事领域的各个角落。从网络防御到武器系统，从情报分析到指挥决策，AI的身影无处不在。然而，这种技术的广泛应用也带来了前所未有的风险。2024年，中美两国领导人就确保核武器始终处于人类控制之下达成共识。今年5月，两国元首又提出建设性战略稳定关系，军事人工智能治理显然应该包含在内。然而，这些政治宣言仍不足以防止AI失控、滥用和冲突升级。中美两国应当探讨如何将原则共识转化为可操作的制度安排，同时识别出当前面临的现实困境。在未来的军事人工智能对话中，还应当纳入管控致命性自主武器系统（LAWS）等更加广泛的议题。

原则如何落地？

中美两国无论在现有的联合声明还是各自的政策主张中都反复强调确保“人类控制”原则。Melanie的文章进一步提出“对核指挥系统和关键基础设施发动网络攻击的决策权只能由人类掌握”。然而，为了避免这些宣言流于形式，中美两国必须在三方面建立务实的合作机制。一是建立军事AI红线清单。明确禁止AI自主发起对NC3系统的攻击以及AI自主决定核武器的使用。同时探讨关键基础设施清单（如金融、医疗、能源等），明确红线的边界范围。

二是就什么是“有意义的人类控制”达成共识。1970年代博依德就提出了著名的OODA理论。后来人们讨论“人在环路”“人类控制”都是在这一理论基础之上。然而在博依德之前，周恩来总理就对中国的核试验工作做出“万无一失”的指示要求。在中国专家和从业者看来，这一指示延续至今，包含了“人类控制”原则。由于双方截然不同的体制机制和战略文化，哪怕对待同一问题也可能使用不同的表述，采取不一样的做法。中美需要通过定期对话或是设立术语工作组的方式来确保彼此接受对方关于“有意义的人类控制”的定义以及相关保障机制。

三是增设中美军事人工智能热线。在现有中美军事沟通机制（如国防部长热线、海上军事安全磋商机制）基础上，增设由于人工智能错误触发或人工智能网络攻击导致的紧急事件通报渠道。目的是降低由于AI故障误导人类甚至僭越人类而导致危机升级的连锁反应风险。

更大的现实挑战

与此同时，双方无需回避地缘政治博弈和技术快速迭代正在加剧安全困境。即便中美就“人类控制”达成合作机制，在现实中仍会面临至少三方面挑战。一是速度困境。在智能体的帮助下，网络攻击正以史无前例的速度展开。AI驱动的防御系统可能自动触发反制措施。设想一次AI检测到NC3系统遭受异常流量攻击可能导致防御AI自动切断网络或启动反制措施。其结果导致对方误判为网络入侵，最终使双方AI系统进入"自动对抗"模式。AI赋能的网络攻击在毫秒级就能完成。在这种超高速对抗中，"人类最终决定"很可能超出了生理极限。

二是溯源困境。网络攻击的溯源极其困难。一次对关键基础设施的攻击，可能是国家行为，也可能是黑客组织、恐怖分子等非国家行为体，甚至是AI自主行为。如果关键基础设施遭到AI驱动的网络攻击，如何判断是政府授权攻击还是非国家行为体滥用AI甚至嫁祸于国家行为体从而挑起冲突？如果没有可靠的溯源机制，"人类责任"就无法落实，报复行动更可能引发错误升级。

三是安全困境。相比传统军事技术，人工智能技术的不透明导致无法准确评估对手AI技术到底发展到什么程度，对军事作战效能的提升效果到底有多大，也无法区分某项军用AI技术到底是进攻性还是防御性的措施。作为一项系统性工程，人工智能的军事应用效果还受到与各个子系统的软硬件耦合、人机协同、军事体制机制以及战略文化等因素的影响。这需要一个较为漫长和复杂的评估过程。结果导致双方经常基于最糟糕的情况来想象对手的AI部署并采取争锋相对的措施。威胁可能被严重夸大，敌意螺旋不断上升。

这些困境在一段时期内难以被彻底消除，只能通过不断加强对话和护栏加以管控。然而，网络攻击只是中美军事AI风险的冰山一角，还有许多重要的议题应当被纳入到两军对话当中。其中，致命性自主武器系统（LAWS）亟待管控。在俄乌战场和中东，自主武器系统的广泛应用已经导致严重的平民伤亡，甚至增加核升级的风险。LAWS一旦落入非国家行为体手中造成的安全风险极大。在联合国《特定常规武器公约》（CCW）就LAWS谈判取得初步进展的情况下，中美应当就"禁止完全自主的致命攻击"和“确保人类控制”寻找最小共识，并围绕如何防止LAWS扩散和被非国家行为体滥用开展对话与合作。在中美关系历史上多次发生由无人载具或装备引发的意外事件。这些装备大多部署在热点地区，一旦与人工智能及进攻性军事能力相结合，很可能引发出乎意料的危机升级。双方还应尽快探讨自主武器系统危机管控和处置预案，共同维护地区和全球战略稳定。

军事AI的治理不是一场可以等待的谈判，而是一场与时间的赛跑。在机器逐渐抢夺“开火权”甚至“交战权”的背景下，中美作为两个AI超级大国，肩负着不让技术失控和守护人类文明的历史责任。期待中美两国元首在9月的会晤中能够延续开展AI对话的积极势头，为全球军事人工智能治理做出表率。

Advancing Human Control of Military AI

Tianjiao Jiang, Associate Professor, Development Institute - Fudan University

Melanie W. Sisson, Senior Fellow - Foreign Policy, Strobe Talbott Center for Security, Strategy, and Technology

The following pieces were authored by members of the U.S. and Chinese delegations who participate in an ongoing Track II Dialogue on Artificial Intelligence and National Security convened by the Brookings Institution’s Foreign Policy program and Tsinghua University’s Center for International Security and Strategy, with support from the Minderoo Foundation. The dialogue, which has been running since 2019, aims to identify practical steps to manage risks relating to artificial intelligence (AI) and national security. The dialogue also undertakes to develop a living glossary of key terms in AI and national security to ensure precision in communication.  

Below, two authors provide a U.S. and Chinese perspective on steps the United States and China could take to build boundaries around acceptable uses of AI in national security. These recommendations, which reflect the authors’ views, are presented in anticipation of a forthcoming U.S.-China government-level dialogue on AI. Together, they explore how the United States and China could build on their existing commitment to maintain human control over nuclear weapons as AI becomes increasingly integrated into the military domain. The authors examine potential areas for further agreement, including ensuring human control over AI-enabled cyberattacks, as well as practical mechanisms and safeguards that could help translate shared principles into practice.

Trump and Xi can lead on governing military AI

Melanie W. Sisson

The United States and China have not developed a common approach to governing AI in the military domain. This does not mean, however, that they are uninterested in addressing the possibility that some military applications of AI will harm civilians, endanger societies, and undermine global stability.

In late 2024, the presidents of the United States and China that humans, not AI, should have control over the decision to use nuclear weapons. This agreement reflects a shared understanding of the catastrophic consequences of nuclear war, and of the need to minimize the likelihood that it will occur not because leaders have chosen it, but because AI fails or is misused.

As important as that agreement is, it is only the starting point: As militaries integrate agentic AI into their operations and AI models advance rapidly in sophistication and capability, new pathways to catastrophic outcomes are emerging. Presidents Donald Trump and Xi Jinping can lead in governing these dangers by agreeing that only humans, not AI, should make the decision to initiate a cyberattack against each other’s nuclear command, control, and communications systems (NC3) or critical infrastructure.

The dangers of AI-powered cyberattacks

States rely on NC3 systems to detect threats, make decisions about nuclear use, and transmit secure, reliable launch orders to the forces that would carry them out. Nuclear doctrine does not eliminate these risks. Whether a state relies on no-first-use, launch-on-warning, or other principles, failures in these functions could: cause breakdowns in leaders’ ability to communicate or issue commands; make a state unable to detect or accurately assess threats; cause misperceptions that incentivize a first strike; or result in mistaken or unauthorized nuclear launch.

An intentional AI-powered cyberattack on another state’s NC3 would therefore initiate a military crisis with possibly catastrophic effects. But an intentional attack is not the only way AI-powered cyber capabilities can cause a military crisis. Such capabilities could also initiate an unintended military crisis in two other ways. First, even if the state’s objective is only to pre-position an offensive cyber capability, an unplanned attack could still result from human error, technical limitations in the AI models, or AI agents that act in ways that humans did not anticipate.

Second, a bad actor could use the AI-powered cyber capability to execute an unauthorized attack. In both cases, the attacked state would be highly likely to respond by elevating alert levels and readying its nuclear forces. If the disruption to normal NC3 functions were severe enough, leaders would have to make decisions about nuclear employment under conditions of extreme time pressure and deep uncertainty, which might result in nuclear launch and war.

AI-powered cyberattacks on critical infrastructure—the economic, financial, health, transportation, and safety systems that support civic life—could also result in an extremely serious military crisis. Such an attack could cause the immediate destruction of property, contamination of the local environment, social chaos, and widespread illness, injury, or death. Like attacks on NC3, attacks on critical infrastructure could be initiated purposefully, or as a result of technical error or unauthorized action. A state that suffers an attack on critical infrastructure would be highly likely to retaliate militarily, initiating a war with unpredictable escalation dynamics.

Presidential leadership is essential

The upcoming U.S.-China AI dialogues are an opportunity for the world’s leading AI countries to meaningfully advance governance of these extremely dangerous uses of AI-powered offensive cyber capabilities. Planned government-to-government dialogues can prepare Trump and Xi to agree that decisions to launch cyberattacks on NC3 and critical infrastructure should be made by humans, not AI.

Extending the principle applied to nuclear weapons—that attacks with strategic effects should express human intent and must be subject to human accountability—does not require Washington and Beijing to share other principles of cybersecurity or infrastructure protection. It is confined to agreement on the narrow point that both sides have the responsibility to minimize the likelihood that attacks with strategic effects will be caused by chance, error, advanced AI capable of exceeding human-defined constraints, or unauthorized actors.

Leader-level statements are the necessary precondition for bilateral dialogue on verifiable protocols and safeguards. They are also important public signals that the United States and China recognize the special responsibilities that come with leading in AI and are committed to taking pragmatic steps to reduce the likelihood that their military uses of AI will put nations at existential risk. No other signal carries the same weight in demonstrating that both governments understand what is at stake—not just for their own societies, but for everyone.

China and the U.S. should accelerate the implementation of “human control”

Tianjiao Jiang

AI is penetrating every corner of the military domain at an unprecedented pace. From cyber defense to weapon systems, from intelligence analysis to command and decisionmaking, AI’s presence is ubiquitous. However, the widespread adoption of this technology also brings with it unprecedented risks. In 2024, the leaders of China and the United States reached a consensus on ensuring that nuclear weapons remain under human control at all times. In May of this year, the two countries’ heads of state further proposed building a constructive strategic stability relationship. The governance of military AI should be an integral part of this effort. Yet these political declarations still fall short of preventing AI from spiraling out of control, being misused, or triggering escalations in conflict. China and the United States should explore how to translate their principled consensus into actionable institutional arrangements, while also identifying the practical challenges currently facing them. In future dialogues on military AI, broader issues—including the regulation of lethal autonomous weapons systems—should also be incorporated.

How can the principles be put into practice?

Both China and the United States have repeatedly emphasized the importance of ensuring the principle of “human control,” as reflected in their existing joint statements as well as in their respective policy positions. Melanie Sisson’s article further argues that the authority to launch cyberattacks against nuclear command systems and critical infrastructure must remain firmly in human hands. However, to prevent these declarations from becoming mere formalities, China and the United States must establish practical cooperation mechanisms in three key areas.

First, they should develop a list of red lines for military AI—clearly prohibiting AI from autonomously launching attacks on NC3 systems or autonomously deciding to use nuclear weapons. At the same time, they should also explore a list of critical infrastructure sectors (such as finance, health care, energy, etc.) and clearly define the boundaries.

Second, they should reach a consensus on what constitutes “meaningful human control.” In the 1970s, United States Air Force Colonel John Boyd proposed the renowned OODA loop—in which whoever can “observe, orient, decide, and act” more quickly in a competition or conflict will seize the initiative.  Later, discussions centered on “human-in-the-loop” and “human control” are based on that theoretical foundation. Yet even before Boyd, Chinese Premier Zhou Enlai had already issued instructions demanding “absolute certainty” (万无一失) in China’s nuclear testing efforts. In the view of Chinese experts and practitioners, this instruction continues to hold sway today and embodies the principle of “human control.” Given the two sides’ fundamentally different institutional frameworks and strategic cultures, even when addressing the same issue, they may employ different terminology and adopt divergent approaches. China and the United States need to ensure mutual acceptance of each other’s definitions of “meaningful human control” and related safeguard mechanisms through regular dialogue or by establishing a terminology working group.

Third, they should establish a China-U.S. military hotline for AI. Building on existing China-U.S. military communication mechanisms—such as the defense minister hotline and the Maritime Military Security Consultation Mechanism—adds a dedicated channel for reporting emergency incidents triggered by AI errors or AI cyberattacks. The goal is to reduce the risk of cascading crises caused by AI malfunctions that could mislead humans or even override human judgment. The 2023 balloon incident should be a wake-up call for both China and the United States. Without the safeguard of a hotline mechanism, AI-driven drones and unmanned vessels are highly likely to trigger more accidental escalations in the future.

Greater real-world challenges

Meanwhile, geopolitics and rapid technological iteration—both of which neither side can afford to ignore—are intensifying the security dilemma. Even if China and the United States were to establish a cooperative mechanism on “human control,” they would still face at least three major challenges in practice. The first is the speed dilemma. With the help of AI agents, cyberattacks are unfolding at an unprecedented pace. AI-powered defense systems could automatically trigger countermeasures. Imagine a scenario where an AI detects abnormal traffic targeting the NC3 system, prompting the defensive AI to automatically disconnect the network or launch countermeasures. As a result, the other side might mistakenly interpret this as a cyber intrusion, ultimately pushing both sides’ AI systems into a mode of “automatic confrontation.” AI-enabled cyberattacks can be carried out within milliseconds. In such ultra-high-speed confrontations, the “final human decision” is likely to exceed physiological limits.

Second, there’s the challenge of attribution. It is extremely difficult to trace the origin of cyberattacks. Critical infrastructure can be targeted by a state actor or by nonstate actors such as hacker groups, terrorists, or even AI agents themselves. If critical infrastructure falls victim to an AI-powered cyberattack, how can we determine whether the attack was authorized by the government or whether a nonstate actor has abused AI—or even deliberately framed a state actor to provoke conflict? Without a reliable attribution mechanism, human accountability cannot be effectively established, and retaliatory actions are all too likely to trigger unintended escalations.

Third is the security dilemma. Compared to traditional military technologies, the opacity of AI technologies makes it impossible to accurately assess the actual level of development of an adversary’s AI capabilities, the extent to which these technologies can enhance military operational effectiveness, or even to distinguish whether a particular military AI technology is offensive or defensive in nature. As a systemic undertaking, the effectiveness of AI’s military applications is further influenced by factors such as the software and hardware integration with various subsystems, human-machine collaboration, military institutional mechanisms, and strategic culture—factors that require a relatively lengthy and complex evaluation process. As a result, both sides often imagine their adversaries’ AI deployments based on the worst-case scenario and adopt countermeasures that are equally confrontational. Consequently, threats may be seriously exaggerated, and the spiral of hostility continues to escalate.

These challenges will be difficult to eliminate completely in the near term and can only be managed through continuous strengthening of dialogue and the establishment of safeguards. However, cyberattacks represent merely the tip of the iceberg when it comes to military AI risks between China and the United States; many other critical issues also deserve to be incorporated into bilateral military dialogues. Among these, lethal autonomous weapons systems (LAWS) urgently require effective control. In the Russia-Ukraine theater and the Middle East, the widespread use of autonomous weapons systems has already led to severe civilian casualties and even heightened the risk of nuclear escalation. Once LAWS fall into the hands of nonstate actors, the security risks they pose are immense.

Given the difficulty in achieving progress on LAWS negotiations within the framework of the United Nations Convention on Certain Conventional Weapons, China and the United States should seek a minimum common ground on “prohibiting fully autonomous lethal attacks” and “ensuring human oversight,” and engage in dialogue and cooperation around measures to prevent the proliferation of LAWS and their misuse by nonstate actors. Throughout the history of U.S.-China relations, there have been numerous accidental incidents involving unmanned vehicles or equipment. Most of these systems are deployed in hotspot regions, and once combined with AI and offensive military capabilities, they could very likely trigger unexpected escalations of crises. Both sides should also promptly explore crisis management and response plans for autonomous weapon systems, working together to uphold regional and global strategic stability.

The governance of military AI is not a negotiation that can afford to wait—it’s a race against time. As machines increasingly seize “the right to fire” and even “the right to engage in combat,” China and the United States, as two AI superpowers, bear the historic responsibility of preventing technology from spiraling out of control and safeguarding human civilization. We look forward to seeing the leaders of China and the United States build on the positive momentum of their ongoing AI dialogue during their September meeting, setting an example for global governance of military AI.

（本文2026年9月9日首发于布鲁金斯学会官网）

原文链接：https://www.brookings.edu/articles/advancing-human-control-of-military-ai/