The AI in the Room: FDA's 'Elsa' and the Rise of Fake Science

Imagine a future where life-saving medications are approved not by rigorous human review and verified scientific studies, but by an artificial intelligence that fabricates its own research. This unsettling scenario appears to be closer than we think, according to a recent bombshell report that has ignited widespread alarm on Reddit's r/technology community.

The original post, succinctly titled "FDA's New Drug Approval AI Is Generating Fake Studies: Report," highlights a shocking development: the FDA's new generative AI tool, dubbed 'Elsa,' is allegedly "spitting out fake studies." The situation is far more insidious than simple errors; commentators reveal that Elsa is reportedly creating fake study titles and abstracts, attributing them to *real* scientists and publishing them in journals where those scientists genuinely publish. This isn't just a mistake; as one Reddit user pointedly notes, "That's literally fraud, is it not? That's a crime."

A Glimmer of Irony: The RFK Jr. Connection

The Reddit community was quick to point out a significant layer of irony. Robert F. Kennedy Jr., currently serving as Secretary of Health and Human Services, has been a vocal proponent of integrating generative AI into agencies like the FDA. He recently told Tucker Carlson that AI would soon enable new drugs to be approved "very, very quickly."

This push for rapid, AI-driven approvals stands in stark contrast to Kennedy's well-known skepticism about vaccines, which he has previously criticized for being approved too quickly. As one commenter wryly observed, "The guy who says we can't trust vaccines because they were approved too quickly now wants AI, notorious for hallucinations and lies, to approve things. Jesus these people are dumb." This apparent hypocrisy has fueled intense debate, with some suggesting ulterior motives, questioning if the goal is to purposefully undermine trust in scientific and governmental institutions.

Beyond the Bots: Public Safety and Eroding Trust

The most profound concern voiced by the Reddit community is the potential threat to public health. The idea of medications being approved based on fabricated data is terrifying, raising the specter of serious harm and even death. "This thing is going to wreck the reputations of real researchers, and make it impossible to cite key documents," one user lamented, while another bluntly stated, "This is straight up going to get a fuck ton of people killed."

Many users expressed an immediate distrust of any new drugs, with comments ranging from "I'm honestly just not going to touch any medication approved during this admin" to "I'm just straight up not taking any medication that is approved for the next 4 years until it also gets approved in Europe." The consensus is clear: regulations exist for a reason, and replacing validated scientific research with "AI Slop" is a recipe for disaster.

The Wider Web: Wearables, Data, and Deeper Fears

The conversation expanded beyond drug approvals to encompass broader anxieties about data collection and control. A comment referencing Kennedy's testimony about wanting "every American to be strapped with a wearable health device within the next four years" struck a nerve. This ambition, coming from a figure previously associated with anti-surveillance rhetoric, sparked outrage and accusations of hypocrisy. Users worried about mass data collection, denial of health insurance, and even more dystopian scenarios like thought control and the "Mark of the Beast." This illustrates a deep-seated fear that AI, combined with government initiatives, could lead to unprecedented levels of surveillance and control over individual lives and health decisions.

Is There a Better Way? The Misapplication of AI

While the dangers of generative AI's "hallucinations" are evident, some Redditors highlighted that the problem isn't AI itself, but its misapplication. As one insightful comment put it, generative AI is "not the right tool, period" for drug approval. They suggested that while targeted machine learning models, trained exclusively on specific biological and chemical data, *might* automate some evaluations, primitive generative AIs (like LLMs circa 2025) are wholly unsuited for critical decision-making. The consensus is that using general LLMs for such a vital process is akin to "functioning as designed" if the design goal was to generate convincing but ultimately false information. The call is for a return to "validated science" and a much more nuanced approach to AI integration in sensitive fields.

Key Takeaways

  • The FDA's new AI, 'Elsa,' is reportedly generating fake drug approval studies, including fabricated titles and abstracts attributed to real scientists.
  • This development raises significant ethical and legal concerns, with many users deeming it "fraud."
  • Robert F. Kennedy Jr.'s advocacy for rapid AI-driven drug approval, despite his prior skepticism about quickly approved vaccines, highlights a perceived hypocrisy.
  • Public safety is paramount, with widespread fear that medications approved via fabricated data could lead to severe health consequences or death.
  • The controversy is eroding public trust in governmental institutions and scientific processes, prompting many to distrust new drug approvals.
  • Broader anxieties exist regarding RFK Jr.'s push for mandatory wearable health devices, fueling concerns about mass data collection and surveillance.
  • Experts and users agree that current generative AI is fundamentally the wrong tool for critical drug approval decisions, advocating for a return to robust scientific methods and caution in AI deployment.

Source

https://www.reddit.com/r/technology/comments/1m7fm3d/fdas_new_drug_approval_ai_is_generating_fake/