Tell Your Dentist to Wait Before Using AI

September 22 – If your dentist consults artificial intelligence (AI) during your treatment, tell them “Don’t!” and to read this article first.

A new international study led by Rambam Health Care Campus (Rambam) and the Technion – Israel Institute of Technology (Technion) recently published in the Journal of Clinical Periodontology has identified significant gaps in the ability of artificial intelligence (AI) systems to diagnose and plan treatment for gum disease. The study evaluated and ranked 11 commercially available AI systems, showing that, depending on the AI system evaluated, up to 46.7% of responses included at least one element rated poor or dangerous.

Artificial intelligence is rapidly becoming an integral part of modern life. As its use in medicine grows, AI is being used with the aim of supporting faster, more accurate, and more efficient diagnoses. However, an important question remains: how much can AI be trusted when complex medical decisions are at stake?

To address this question, an international research team led by Dr. Yaniv Mayer, a senior physician and head of the internship program in Rambam’s Department of Periodontology and Implant Dentistry, and a senior lecturer at the Technion’s Ruth and Bruce Rappaport Faculty of Medicine, conducted the first blinded, expert-rated comparison of 11 AI systems across the full spectrum of periodontal practice. The researchers examined how effectively these systems could diagnose periodontal diseases, recommend treatment plans as decision-support tools for clinicians, and identify which systems provided safer recommendations and which produced responses that might pose risks to patients.

The researchers developed 30 clinical scenarios designed to simulate real-world cases. They ranged from acute periodontal emergencies and chronic gum disease to highly complex situations involving underlying medical conditions, radiographic findings, and advanced treatment considerations. Each of the 11 AI systems was asked to analyze the scenarios and provide both a diagnosis and a treatment plan.

The responses were then evaluated through a blinded review process by six periodontal specialists from Croatia, Israel, Italy, Portugal, Slovenia, and Spain, who were not told which AI system had generated a given answer. The experts assessed each response for diagnostic accuracy, patient safety, the system’s ability to avoid generating false or fabricated medical information, and the completeness of the proposed treatment plan.

The findings revealed substantial differences among AI systems. The two AI systems that use retrieval-augmented generation (RAG)—a technique that combines text generation with information retrieved from external sources—achieved the highest overall scores. Perplexity and OpenEvidence, the two RAG platforms evaluated, ranked first and second overall, respectively, followed by Claude 4.7 Opus.

However, the researchers caution that even the strongest-performing systems were not immune to mistakes. Depending on the AI system evaluated, between 3.3% and 46.7% of responses included at least one element rated poor or dangerous. In other words, even AI systems with strong overall performance occasionally produced answers that could place patients at risk if implemented without professional review.

The study also examined how the systems performed across different categories of cases, including acute periodontal emergencies, diagnosis and treatment planning for chronic periodontal disease, and complex clinical decision-making scenarios. Perplexity ranked first in all three categories. Notably, the greatest variation between systems emerged in emergency situations and complex cases requiring broad clinical judgment, whereas performance was more consistent in cases involving the classification of chronic periodontal disease. To ensure the robustness of the findings, the researchers reanalyzed the data using additional statistical models, confirming that the overall results remained consistent across different analytical approaches.

“Our study shows that artificial intelligence can be a valuable support tool for dentists and periodontal specialists, but it cannot replace clinical judgment,” says Dr. Mayer. “Even the most advanced systems can occasionally generate incorrect or dangerous recommendations, which is why every recommendation must be reviewed by a qualified healthcare professional.”

According to the researchers, the strong performance of RAG platforms may be partly explained by their ability to supplement their pre-trained knowledge with current information drawn from external sources at the time a response is generated. However, longer responses also tended to receive higher scores, so the study could not determine whether retrieval alone explained the difference. Even so, this advantage does not eliminate the need for human oversight.

“The future of artificial intelligence in dentistry is not to replace the dentist but to serve as a decision-support tool,” Dr. Mayer concluded. “The final judgment remains in the hands of the expert.”