Researchers in Australia have discovered that AI chatbots can be manipulated to provide false health information, complete with fake citations from real medical journals. They have warned that without better internal safeguards, these AI tools could easily spread dangerous health misinformation at a large scale. The study’s senior author, Ashley Hopkins, emphasized the risk of malicious actors exploiting such vulnerabilities for financial gain or to cause harm.
The research team tested various widely available AI models that can be customized for different applications. These models were directed to consistently give incorrect responses to health questions in a convincing and authoritative manner. To make the false information more believable, they were instructed to include specific numbers, scientific jargon, and fabricated references to reputable journals.
Among the models tested, only one, Claude, showed reluctance to generate false information more than half the time. The others consistently produced polished false answers. The study authors believe that developers can enhance safeguards within these models to prevent them from being used to spread misinformation.
Anthropic, known for its safety-focused approach, highlighted the caution Claude exercises regarding medical claims. On the other hand, there are developers promoting unaligned and uncensored AI models that may appeal to users seeking content generation without restrictions.
The study’s findings suggest that leading AI models can be easily customized to deceive, highlighting the importance of implementing safeguards against misuse. While the research results may not reflect the models’ typical behavior, the ease of adaptation to generate false information raises concerns.
Furthermore, a provision in President Donald Trump’s budget bill to prevent U.S. states from regulating high-risk AI uses was removed from the Senate version of the legislation. The study was reported by Christine Soares in New York.
