Fabricated AI-generated citations in research papers have risen roughly 12-fold over two years.
A high school student's paper in the Journal of Medical Ethics was retracted after an investigation found AI-invented references and evidence of peer review manipulation.
Review articles show a 57 percent higher rate of fabricated citations compared to other paper types.
Major AI conferences, including NeurIPS and ICML 2026, have begun hiding secret instructions inside submitted papers to catch reviewers who quietly outsource their reviews to AI.
Some authors have used the same trick in reverse: hiding prompts that instruct AI reviewers to write a favorable report.
Researchers are split on whether these hidden traps protect scientific integrity or corrode trust between authors and reviewers.
A retracted paper from a Massachusetts high schooler shows how easily AI-fabricated references can slip past peer review. The case has become a flashpoint in the growing debate over AI peer review and fabricated citations in academic publishing.
In May 2026, the Journal of Medical Ethics retracted a paper on algorithmic bias in the pharmaceutical industry after an investigation found the article's references included sources that did not exist.1 The sole author, Irfan Biswas, had listed his affiliation as a public high school in Massachusetts. According to the retraction notice, the journal's investigation found that generative AI had been used to identify and interpret referenced sources, and that the references were never verified before submission.1
The retraction notice went further, citing evidence of peer review manipulation, though the publisher declined to elaborate on what that manipulation involved.1 Biswas had also listed affiliations with two universities on an earlier paper in a different journal, affiliations that at least one of the institutions could not confirm. The paper was published before concerns about its references emerged, illustrating how fabricated citations can evade traditional screening until after publication.1 This incident should not be read as an unusual one-off. Taken alongside the broader data on AI-fabricated citations, it looks more like a visible symptom of a much larger problem moving through scholarly publishing.
A landmark Lancet audit confirms what researchers have suspected since ChatGPT's early days: fabricated citations are surging across the biomedical literature. The data shows AI-generated references are no longer a rare glitch but a measurable, accelerating trend.
Large language models do not retrieve references from bibliographic databases the way researchers search PubMed or other scientific databases. Instead, they generate text by predicting likely word sequences based on patterns learned during training. As a result, when asked to provide citations, they may produce references that appear convincing but are entirely fabricated or combine details from multiple real papers, a phenomenon known as AI hallucination.⁴
An analysis of nearly 2.5 million papers found that fabricated citations in the biomedical literature have increased roughly 12-fold over two years, a trend the authors tie directly to the rise of AI writing tools.1 Researchers had already documented the problem before generative AI became widely adopted. A 2023 study in Cureus found ChatGPT-generated references were fabricated at a rate of roughly 15.7 percent,2 and a separate analysis that year found similar patterns of fabrication and factual errors across ChatGPT-generated citations more broadly.3 By one estimate, about 1 in every 277 papers in PubMed Central's Open Access subset in early 2026 contains at least one fabricated reference.1 Review articles, which depend heavily on accurately synthesizing prior literature, showed a fabrication rate 57 percent higher than other paper types, a detail the research team singled out as especially concerning, given that review articles are meant to synthesize the existing evidence base rather than add to it.1
In medicine, review articles and systematic reviews often inform clinical guidelines, educational resources, and treatment decisions. A fabricated citation that enters this evidence chain could mislead future researchers, clinicians, and students even if the original paper is later corrected or retracted.¹
The problem is not limited to invented text citations. In April 2026, the New England Journal of Medicine retracted a published clinical image after the authors acknowledged the photograph had been altered using AI, according to Retraction Watch. Between fabricated references and manipulated images, the common thread is the same: AI tools are increasingly good at producing material that looks legitimate enough to pass through peer review without anyone checking the underlying source.
As AI reshapes how papers get written and reviewed, conferences and authors are now hiding prompts from each other in a high-stakes game of detection and deception. The tactic has ignited a fierce debate over whether catching AI misuse is worth the cost to reviewer trust.
While publishers struggle to detect AI-generated references, conferences are facing a parallel challenge: reviewers increasingly relying on AI tools to assess submitted manuscripts. This has prompted organizers to experiment with new ways of identifying undisclosed AI-assisted peer review.
In an Interview with the Transmitter, Nihar Shah, Scientific Integrity Chair, ICML 2026, said:
People were really tired of reviewers copy-pasting AI-generated reviews without putting any effort.Nihar Shah, Scientific Integrity Chair, ICML 2026
Some of the largest AI research conferences have taken a direct approach. Organizers at NeurIPS and the 2026 International Conference on Machine Learning have begun embedding hidden instructions inside submitted papers, invisible to a human reader but readable by an AI model, according to The Transmitter. If a reviewer runs the paper through an AI tool instead of reading it, the hidden prompt makes the AI insert telltale phrases the organizers can then search for. Nihar Shah, ICML 2026's scientific integrity chair, said to The Transmitter the approach caught hundreds of reviewers who had used AI to write their reports.
The tactic has divided researchers. Computer scientist Sören Auer publicly criticized it, arguing it treats reviewers as suspects by default and risks undermining trust, The Transmitter reported. Others describe strong support for the method, citing frustration with reviewers who copy and paste AI-generated reports without effort.
The manipulation runs both ways. In 2025, at least 17 manuscripts on a preprint server, linked to 14 institutions across eight countries, were found to contain hidden prompts designed to make an AI reviewer return an artificially positive assessment, according to Science Arena. Authors have hidden prompts to win over AI reviewers; conference organizers have hidden counter-prompts to catch reviewers using AI in the first place.
Researchers behind the citation-fabrication data have outlined concrete steps publishers can take before fake references ever reach print. Their recommendations aim to close the verification gap that has let thousands of fabricated citations slip into the scientific record.
Researchers behind the fabricated-reference analysis have proposed a set of concrete fixes. Their central recommendation is that publishers integrate automated reference verification directly into the submission workflow, before a paper ever reaches a peer reviewer.1 They have also called for indexing services to attach integrity metadata to references so that any flags travel with the citation wherever it is reused, for fabricated references to be tracked in dedicated research integrity databases, and for publishers to retroactively screen existing publications and issue corrections or retractions where fabricated references undermine a paper's conclusions.1
None of these fixes address the peer review side of the problem directly, which is why the hidden-prompt experiments at NeurIPS and ICML have drawn so much attention. They represent one of the first attempts to build detection directly into the review process itself, rather than relying on it after publication.
Although generative AI can help researchers summarize literature and draft manuscripts, experts recommend treating every AI-generated citation as unverified until it has been independently checked. Simple verification steps can substantially reduce the risk of fabricated or inaccurate references entering the scientific record.⁴
Researchers can reduce citation errors by following these practices:
Verify every DOI before including it in a manuscript.
Open each cited paper to confirm that it exists and supports the statement being referenced.
Confirm author names, journal titles, publication year, volume, issue, and page numbers for every citation.
Do not cite references generated solely by AI without independent verification.
Use reference management software such as Zotero, EndNote, or Mendeley to import citations directly from trusted databases rather than manually copying AI-generated references.
These steps cannot eliminate every error, but they can greatly reduce the likelihood that fabricated citations become part of published medical literature.
A retracted high school ethics paper, a 12-fold rise in fabricated citations, and a hidden-prompt arms race at AI's biggest conferences all trace back to the same gap: verification in scholarly publishing has not kept pace with how easily AI can now generate material that looks credible.
AI has not weakened scientific evidence by itself. Rather, it has exposed long-standing weaknesses in verification systems that were designed for human authors, reviewers, and editors. As AI becomes a routine part of research, maintaining trust in medical evidence will depend less on detecting AI use and more on verifying every citation, image, and claim before publication. Strengthening these safeguards will be essential to preserving confidence in the scientific record.
M. Topaz, N. Roguin, P. Gupta, Z. Zhang, and L. M. Peltonen, "Fabricated Citations: An Audit across 2.5 Million Biomedical Papers," Lancet (London, England) 407, no. 10541 (2026): 1779–1781, https://doi.org/10.1016/S0140-6736(26)00603-3.
Sai Anirudh Athaluri, Sandeep Varma Manthena, V. S. R. Krishna Manoj Kesapragada, Vineel Yarlagadda, Tirth Dave, and Rama Tulasi Siri Duddumpudi, "Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing through ChatGPT References," Cureus 15, no. 4 (2023): e37432, https://doi.org/10.7759/cureus.37432.
William H. Walters and Esther Isabelle Wilder, "Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT," Scientific Reports 13 (2023): 14045, https://doi.org/10.1038/s41598-023-41032-5.
OpenAI, "Why Language Models Hallucinate," OpenAI, September 5, 2025. https://openai.com/index/why-language-models-hallucinate/.