MIT study: AI helps detect fakes short-term, harms unaided skill

Researchers at MIT’s Media Lab tracked 67 people over four weeks to measure how conversational AI affects the ability to spot false news. During sessions with an AI assistant, accuracy on headline-and-image tasks rose about 21% compared with unaided judgments. When participants were later tested on new, unseen items without AI help, their accuracy dropped by 15.3 percentage points from the baseline.

The drop in performance was driven mainly by a weaker ability to identify fake items; accuracy on real news remained essentially unchanged. The study logged 7,203 conversations between users and the assistant and recorded 4,536 individual judgments about the authenticity of news headlines and images.

Participants first made an unaided judgment about a headline and image, then consulted a system that combined OpenAI’s GPT-4o with Google Search, and finally made a revised judgment. Researchers later tested participants on different items without AI to measure any lasting change in skills.

To analyze patterns in how people interacted with the assistant, the team used Anthropic’s Claude 3.5 Sonnet to review thousands of exchanges. The paper notes the experiments used earlier model versions and says it is unclear whether newer systems with stronger reasoning would produce different results.

The paper states, “Current approaches prioritize belief correction over skill development, creating dependency rather than durable discernment capabilities.” The authors describe the finding as a reduction in independent ability to flag false claims after relying on AI assistance.

The study was published as social platforms respond to an increase in AI-generated content. Following missile strikes in June 2025, videos claiming to show destruction in Tel Aviv and at Ben Gurion Airport circulated widely before being identified as AI-generated. Some platforms have taken steps to limit undisclosed AI-generated conflict footage and to penalize creators who post such material without clear disclosure.

The researchers report the measured short-term benefit during AI use alongside a later decline in unaided detection on new items. The paper says questions remain about whether different training approaches, longer exposure, or more advanced models would change people’s unaided accuracy.