AI Flags More Than 250,000 Suspicious Cancer Research Papers with 91% Accuracy
A large study published in The BMJ found that of 2.6 million cancer research papers released between 1999 and 2024, more than 250,000 contained writing patterns similar to those seen in studies already retracted over suspected fabrication. The research team trained a BERT language model to detect the output of “paper mills,” businesses that sell fake or low-quality studies on an industrial scale, and it correctly identified suspicious papers 91 percent of the time. The team calls the system a “scientific spam filter,” and three journals are already testing it.
- The study examined 2.6 million cancer papers from 1999 to 2024 and found over 250,000 with writing patterns resembling retracted, fraudulent work
- A BERT language model trained to spot paper mill textual “fingerprints” detected suspicious papers with 91 percent accuracy
- Flagged papers rose from about 1 percent in the early 2000s to a peak of over 16 percent in 2022; three journals are trialing the tool
The paper mill problem is larger than realized
The study was led by Professor Adrian Barnett of the Queensland University of Technology (QUT) with an international group of collaborators. Paper mills sell authorship positions and, in some cases, complete ready-made scientific papers. These studies may contain reused text, unusual or awkward language, and fabricated data or images. Barnett says paper mills most likely rely on boilerplate templates, which can be detected by language models that analyze patterns in text.
A “scientific spam filter” with 91% accuracy
The team trained a BERT model to identify subtle textual “fingerprints” that repeatedly appear in known paper mill products. When evaluated using verified examples, the model correctly detected suspicious papers 91 percent of the time. Barnett compares it to email spam filtering: the tool flags papers matching the writing style and structure seen in retracted, fraudulent work.
Trends and applications
The large-scale analysis revealed several major trends. Flagged papers rose sharply over the past two decades, from about 1 percent in the early 2000s to a peak of more than 16 percent in 2022. Suspicious papers appear across thousands of journals published by major companies, including journals with strong reputations and high impact. The problem is especially common in molecular cancer biology and early-stage laboratory research, with gastric, liver, bone and lung cancer studies flagged at particularly high rates. Three scientific journals are already testing the system in their editorial review process, and the team plans to adapt it for other scientific fields. The researchers emphasize that flagged papers should not automatically be treated as fraudulent: the results are warning signals, not confirmed findings of misconduct, and each case still needs review by human experts.
Why it matters
Barnett notes that cancer research influences clinical trials, drug development and patient care. If fabricated studies make their way into the evidence base, they can mislead real scientists and ultimately slow progress for patients, which is why getting ahead of the problem is vital.
Source: ScienceDaily ยท Study published in The BMJ
