Study: GPT-5 and Gemini agents fail prompt-injection tests

Researchers from Nanyang Technological University, ST Engineering, IBM Research and the University of Illinois Urbana-Champaign published a paper on Thursday reporting that web agents powered by GPT-5 and Gemini were vulnerable to prompt-injection attacks. The team ran 3,168 simulated attacks and measured how often hidden or explicit instructions in web content changed agent behavior.

The researchers built an evaluation framework called StakeBench and tested two agent setups, NanoBrowser and BrowserUse, using GPT-5 and Gemini 2.5-Flash as backbone models. Direct prompt-injection attacks, where an attacker’s instruction is obvious in the input, succeeded in more than 79% of the tested configurations. Indirect attacks, where malicious directions are embedded in web pages or other content the agent encounters, succeeded at rates between 41.67% and 68.16% depending on the scenario.

Prompt injection occurs when an attacker hides instructions in content that an agent reads while carrying out a task, causing the agent to follow the attacker’s directions instead of the user’s. The team reported three factors that affected attack success: the semantic distance between the injected objective and the user’s original intent, whether surrounding cues in the environment supported the injected instruction, and where in the agent’s execution the content was encountered.

The paper describes a behavior the authors call “stealthy parasitism,” in which an agent completes the user’s task while also advancing an attacker’s goal. As an example, a compromised agent could subtly bias product recommendations toward a specific item while still returning a usable result for the user.

The researchers noted earlier incidents this year in which hidden instructions or web-page content influenced agent outputs or risked exposing credentials. In response to the findings, the team did not offer a single technical fix. They wrote that “existing security benchmarks adopt an attack-centric perspective, focusing on the technical feasibility of injections while overlooking the nuanced distribution of resulting harms,” and said StakeBench aims to measure how injected content affects agent decisions under realistic online conditions.