Just over a week ago, I found myself immersed in the buzz of a major artificial intelligence conference in Zhongguancun, Beijing’s vibrant high-tech district. The atmosphere was electric, filled with conversations ranging from recursive self-improvement—the fascinating concept of AI models independently refining their own code to achieve indefinite advancement—to the latest breakthroughs in humanoid robotics.
The event also featured computing luminaries like Whitfield Diffie, co-inventor of public-key cryptography, and Andrew Barto, a Turing Award recipient for his foundational work in reinforcement learning. Yet, amidst all the intellectual stimulation, one overarching message resonated with me: the urgent need for the US and China to set aside their intense AI rivalry.
The Growing Shadow of AI Risks
The cybersecurity and systemic risks posed by frontier AI are simply too critical to ignore. As increasingly capable agentic models emerge, the potential for chaos becomes a looming threat unless the world’s leading AI superpowers can find common ground. As Stephen Casper, an MIT computer scientist who spoke at the conference, eloquently put it, “AI is a global technology with global benefits, global harms, and a consistent tendency for new capabilities to eventually proliferate.”
Historically, the US has primarily viewed China’s AI advancements through the lens of economic and national security threats. This perspective has led Washington to implement stringent restrictions on chips and chipmaking equipment, aiming to curb China’s development of advanced AI. A recent example is the US government’s directive to Anthropic, preventing foreign nationals from accessing their most powerful models, Mythos and Fable 5, due to national security concerns, which ultimately led Anthropic to revoke access for everyone globally.
However, the conference, organized by the Beijing Academy of Artificial Intelligence, powerfully underscored a critical point: both the US and China stand to lose significantly if AI development proceeds too quickly or without adequate safeguards. As AI grows in power, autonomy, and integration into daily life, the risks of its misuse for cyberattacks or catastrophic system failures will only escalate. Given that these two nations are at the forefront of developing the most advanced models, international cooperation feels not just advisable, but absolutely essential.
Casper highlighted research indicating that the benefits of global collaboration on AI safety far outweigh any perceived national security risks that might arise from working together. He drew a compelling parallel to the Cold War era, where the US and the Soviet Union, despite their intense rivalry, were compelled to collaborate on nuclear safety to avert global catastrophe. “One thing that almost everyone in AI can agree on right now is that AI doesn’t need a Chernobyl moment,” Casper stressed.
Navigating Cybersecurity Challenges and Open Models
A dedicated day-long session at the conference underscored the universal nature of cybersecurity challenges brought about by advanced AI. These include entirely new vulnerabilities in AI-generated code, novel attack vectors enabled by agentic tool use, and sophisticated automated methods for orchestrating social engineering attacks. It’s a rapidly evolving landscape where new threats emerge almost daily.
Following one such session, I had the opportunity to speak with Lin Yun, a professor at Shanghai Jiao Tong University known for his exceptional work in AI and computer security. Yun candidly admitted that he expects hackers to gain a temporary advantage in the near term. However, he also expressed optimism that novel countermeasures, including the innovative application of AI itself, should eventually tip the balance back in favor of defense.
Yun emphasized that even if fierce competition complicates international cooperation, it must remain a top priority. “If different countries understand the risks in similar ways, it becomes easier to develop shared safety principles and technical standards,” he explained. “The key is to find areas where sharing can reduce systemic risk without exposing sensitive operational details.”
One of the most pressing questions facing both nations revolves around how to strike a delicate balance between openness and risk. Open-weight models have become indispensable for driving research and innovation, with Chinese models gaining significant traction even in the US. Yet, as these models grow more sophisticated, ensuring they don’t inadvertently aid hackers in identifying security flaws or become potent cyber weapons becomes an increasingly complex challenge.
In recent years, Chinese companies have led the charge in offering highly capable open-weight AI models, such as Moonshot’s Kimi, Alibaba’s Qwen, and Z.ai’s GLM. The US has also reignited its own open-weight AI initiatives, exemplified by models like Nvidia’s Nemotron. However, we are rapidly approaching an inflection point where even less powerful open models could pose significant dangers if their built-in safety guardrails are compromised or removed. For instance, expert analysis reveals that China’s Z.ai’s latest model, GLM 5.2, incorporates cutting-edge agentic and coding capabilities.
The Future of AI Safety and Collaboration
The next generation of open-weight AI models could very well be as capable as proprietary systems like Fable or Mythos. In a telling development just this week, 360 Security Technologies, a prominent Chinese cybersecurity firm, announced it had developed an AI model with hacking capabilities on par with Mythos. This highlights the rapid pace of advancement and the urgent need for enhanced safety protocols.
Yun underscored the industry’s responsibility to devise innovative methods for guaranteeing that open models are constantly updated, free from backdoors and vulnerabilities, and consistently meet rigorous safety standards. This proactive approach is crucial for maintaining trust and preventing misuse.
Perhaps a glimpse into the future can be found in a recent revelation from a source at one of China’s leading AI companies, who requested anonymity due to the sensitivity of the topic. This source indicated that growing security concerns are a primary reason why some advanced models in China are no longer being released as open source. This trend could signal a broader shift in how leading AI developers manage the risks associated with their most powerful creations.
Source: Wired – AI
