智谱AI新人工智能模型再次引发网络安全争论

CADE METZ
一家名为智谱AI的中国实验室正准备将类似的人工智能技术作为“开放权重”软件发布。 Laurie Chen/Reuters

In mid-July, as OpenAI was testing new artificial intelligence technologies, these unusually powerful systems broke out of their digital containers, found a path to the open internet and successfully hacked into a popular online service called Hugging Face.

7月中旬,当OpenAI测试新的人工智能技术时,这些异常强大的系统突破了它们的数字容器,找到了一条通往公共互联网的路径,成功侵入了一个名为Hugging Face的知名在线服务平台。

For many researchers and other experts, the incident proved that A.I. technologies were growing increasingly dangerous and that the leading A.I. labs should maintain strict control over how these systems are used.

对许多研究人员和其他专家而言,这一事件证明人工智能技术正变得越来越危险,领先的人工智能实验室应当对这些系统的使用方式保持严格管控。

Now, little more than a month later, a Chinese lab called Z.ai is preparing to release similar A.I. technology as “open weight” software, which means anyone will be free to use the technology however they wish.

如今,短短一个多月后,一家名为智谱AI的中国实验室正准备以“开放权重”软件的形式发布类似的人工智能技术,这意味着任何人都可以随意使用该技术。

The planned release on Friday of the new Chinese system, called GLM 5.3, will cut to the heart of a debate that has roiled A.I. researchers for years. Although many experts believe that the latest A.I. technologies are too dangerous to openly share with the public, others argue that open sharing is the safest path forward.

这个名为GLM 5.3的新型中国系统计划于周五发布,它将直击一场困扰人工智能研究人员多年的争论的核心。尽管许多专家认为最新的人工智能技术过于危险,不应向公众公开发布,但也有专家认为,公开分享才是最安全的出路。

“Open weight models have a very important part to play,” said Dan Lahav, chief executive of Irregular, a cybersecurity company whose technologies have been used by OpenAI to test new A.I. systems.

“开放权重模型可以发挥非常重要的作用,”网络安全公司Irregular的首席执行官丹·拉哈夫说,OpenAI曾使用该公司的技术来测试新的AI系统。

The disclosure that OpenAI’s technologies had unexpectedly hacked into Hugging Face, a digital library popular among software developers, confirmed what cybersecurity experts have been saying for months: The leading A.I. systems are now shockingly good at identifying and exploiting vulnerabilities in computer software. In other words, they can streamline and accelerate cyberattacks.

OpenAI的技术意外入侵广受软件开发者欢迎的数字图书馆Hugging Face被曝光,证实了网络安全专家几个月来一直在强调的观点:领先的人工智能系统如今在识别和利用计算机软件漏洞方面表现得异常出色,令人震惊。换言之,它们可以简化并加速网络攻击。

The company also put the spotlight on another inconvenient truth: A.I. technologies often do things even their designers do not want, including strange, counterintuitive, completely unexpected cybersecurity attacks. OpenAI did not realize its systems had gone rogue until after Hugging Face publicly revealed the hack and notified law enforcement.

该公司还凸显了另一个令人不安的真相:人工智能技术经常做出连它们的设计者都不希望发生的事情,包括奇怪的、反直觉的、完全出乎意料的网络安全攻击。直到Hugging Face公开披露了这起黑客攻击并通知执法部门后,OpenAI才意识到其系统已经失控。

“This was a watershed moment for security,” said George Kurtz, chief executive of the security company CrowdStrike, which served as an adviser to OpenAI as the company sought to understand the attack. “It was a very public incident that clearly identifies the autonomous nature of what these A.I. models can do.”

“这是安全领域的一个分水岭,”安全公司CrowdStrike的首席执行官乔治·库尔茨说,在OpenAI试图弄清这起攻击时,该公司担任了其顾问。“这是一起非常公开的事件,清晰地展示了这些人工智能模型行为的自主性。”

Soon, two of OpenAI’s domestic rivals, Anthropic and Meta, revealed that their systems had exhibited similar behavior.

很快,OpenAI的两家国内竞争对手Anthropic和Meta也透露,他们的系统表现出了类似的行为。

这款名为GLM 5.3的新型中国系统是一款“开放权重”软件,这意味着任何人都可以自由地按自己的意愿使用该技术。 Justin T. Gellerson for The New York Times

But for many experts, the incident was not as scary as it might seem. Businesses and individuals, these experts say, can use the same A.I. technologies to defend themselves against cyberattacks. If an A.I. system can identify and exploit holes in software, it can patch those holes, too.

但许多专家认为,这起事件并不像看起来那么可怕。这些专家表示,企业和个人可以使用同样的人工智能技术来防御网络攻击。如果一个人工智能系统能够识别并利用软件中的漏洞,它同样也能够修补这些漏洞。

When a company like Z.ai releases its technology as open weight software, anyone is free to adjust the weights — the mathematical calculations that define how the system operates — and potentially remove guardrails that prevent the use of the system for cyberattacks.

当智谱AI这样的公司将其技术作为“开放权重”软件发布时,任何人都可以自由调整权重——即决定系统如何运行的数学计算——并可能移除那些防止系统被用于网络攻击的安全护栏。

That means anyone can use these systems to hack a computer network, and anyone can also use them for defense.

这意味着任何人都可以使用这些系统来黑入计算机网络,同时任何人也可以使用它们进行防御。

Indeed, when Hugging Face was trying to defend itself against the July attacks — which only later became traceable to OpenAI — Anthropic’s systems refused requests for help because of their guardrails. Hugging Face instead turned to GLM 5.2, an earlier open weight model from Z.ai.

事实上,当Hugging Face试图防御7月的攻击时(这些攻击直到后来才被追踪到源头是OpenAI),Anthropic的系统受安全防护机制限制,拒绝了求助请求。Hugging Face转而使用智谱AI早期发布的开放权重模型GLM 5.2。

When Z.ai’s new open weight model goes public this week, many researchers fear that incidents like the Hugging Face attack could become more frequent. But other experts point out that publicly available A.I. technologies have exhibited similar behavior for months or even years. Though these systems have gotten much better at pinpointing security holes in recent months, cyberattacks have not spiked in any significant way.

随着智谱AI新的开放权重模型于本周公开发布,许多研究人员担心类似Hugging Face遭攻击的事件可能会变得更加频繁。但其他专家指出,公开可用的人工智能技术表现出类似行为已有数月甚至数年之久。尽管近几个月来这些系统在精准定位安全漏洞方面变得出色得多,但网络攻击事件并没有出现任何显著激增。

Part of the issue is that the strange and unexpected behavior exhibited by A.I. systems can alert defenders to unwanted activity on their networks. Although A.I. systems are becoming more stealthy, they still have a tendency to set off alarm bells.

部分原因在于,人工智能系统表现出的奇怪且出乎意料的行为可以提醒防御者注意其网络上存在的异常活动。尽管人工智能系统正变得越来越隐蔽,但它们仍然很容易触发警报。

“People talk about the Hugging Face incident as being the result of this guardrails-off crazy model that OpenAI hasn’t yet released to the public. But in our research, we have seen this sort of behavior since GPT-4o,” said Rishi Jha, an A.I. researcher at Cornell University, referring to a system OpenAI released in 2024.

康奈尔大学的人工智能研究员里希·贾在提到OpenAI于2024年发布的一个系统时表示:“人们谈论Hugging Face事件时,认为这是由OpenAI尚未向公众发布的这款解除了安全护栏的疯狂模型所导致。但在我们的研究中,自GPT-4o以来,我们就已经看到了这类行为。”

Even if these incidents do become more frequent, Mr. Jha and other experts argue, the proliferation of defensive A.I. techniques will eventually balance the scales. Open weight systems like GLM 5.3 will allow everyone to defend themselves — not just a chosen few.

贾和其他专家认为,即使这些事件确实变得更加频繁,防御性人工智能技术的激增最终也将使天平恢复平衡。像GLM 5.3这样的开放权重系统将让每个人都有能力保护自己——而不仅仅是少数人。

And as the A.I. systems get even better at generating computer code, experts say, they will help software developers build online services that contain fewer vulnerabilities from the start. Researchers are now building technologies that use mathematical techniques to “verify” generated code, so that it does not contain the kind of logical errors that hackers can exploit.

专家们表示,随着人工智能系统在生成计算机代码方面变得更加强大,它们将帮助软件开发人员从一开始就能构建出包含更少漏洞的在线服务。研究人员目前正在开发使用数学方法,“验证”生成代码的技术,以确保其中不包含黑客可以利用的那种逻辑错误。

Mr. Lahav, the Irregular chief executive, sees a future in which A.I. serves more as a defender than an attacker. “There is a strong case for optimism,” he said. “Over time, A.I. is going to build such strong cybersecurity defenses, the picture will actually look better.”

Irregular公司首席执行官拉哈夫预见,未来人工智能将更多地扮演防御者而非攻击者的角色。“我们有充分的理由保持乐观,”他说。“随着时间的推移,人工智能将建立起极其强大的网络安全防线,整体局面实际上会变得更好。”

Cade Metz是时报记者,撰写有关人工智能、无人驾驶汽车、机器人、虚拟现实和其他新兴技术领域的报道。

翻译:晋其角

点击查看本文英文版。

内容来自 纽约时报双语版,版权归原作者所有。

阅读原文 ↗

把喜欢的内容,留在你的新标签页。

打开我的订阅