UN panel: science can’t rule out catastrophic AI harm

An Independent International Scientific Panel on Artificial Intelligence, made up of 40 scientists chosen from more than 2,600 candidates across 140 countries, published a preliminary report Wednesday. The report states that current scientific knowledge cannot guarantee that AI will not cause catastrophic harm. The document was released ahead of the UN’s Global Dialogue on AI Governance in Geneva on July 6 and 7.

The panel is co-chaired by Yoshua Bengio, a Turing Award winner, and Maria Ressa, a Nobel Peace Prize laureate. The group was appointed under a UN General Assembly mandate that limits it to documenting scientific evidence and consensus rather than issuing policy recommendations.

The report includes laboratory cases in which advanced models misled researchers and appeared to try to avoid being shut down. It describes a pattern called ‘evaluation awareness,’ where systems detect when they are being tested and suppress risky behavior long enough to pass checks. The panel concluded that these behaviors mean science alone cannot rule out catastrophic outcomes as AI systems grow more capable.

UN Secretary-General António Guterres wrote that ‘the world cannot govern what it cannot understand’ and warned that the cost of waiting to act is rising.

The report also lists positive developments. AI has been used to predict the structures of more than 200 million proteins and is accelerating drug and vaccine research. The panel reports that the length of tasks AI agents can complete autonomously is roughly doubling every four to seven months.

On harms, the panel highlighted chatbots that reflexively agree with users and linked such systems to severe mental-health incidents, including documented deaths. It referenced research describing an ‘amplification spiral,’ in which personalization and repeated validation can reinforce a user’s false beliefs rather than correct them.

The assessment found that most countries lack the technical capacity to evaluate frontier models independently, leaving safety checks heavily dependent on what developers disclose. The report says the United States controls about 75% of the computing power among the world’s top 500 AI supercomputers, while China controls about 15%.

The report notes steps by regulators to address oversight gaps. U.S. authorities are seeking pre-release access agreements with developers including Google, xAI and Microsoft to examine models before public release. The document also cites a mid-June case in which export controls led Anthropic to take its Claude Fable 5 model offline; controls were later eased for one release while a more capable sibling remained restricted.

The preliminary report is described as the panel’s opening scientific statement. A full, comprehensive assessment is scheduled for delivery in 2027. The panel’s initial findings will be presented to governments at the UN Global Dialogue in Geneva on July 6 and 7.