LLMs are powerful. But are their results reproducible enough for clinical decision support? 🤖🩺
Our ECIR 2026 demo paper, “MedNuggetizer: Confidence-Based Information Nugget Extraction from Medical Documents”, addresses this question.
🧗 The challenge
Large language models open up powerful opportunities for automated evidence synthesis in medicine but they also introduce a critical issue: reproducibility.
The same prompt can produce different outputs across runs due to stochastic sampling and model variance. In clinical decisions, this instability is risky.
💡 Our solution: MedNuggetizer
MedNuggetizer helps clinicians explore reliable medical evidence from multiple long documents.
It is a query-driven tool that repeatedly extracts and clusters “information nuggets” from medical documents, estimating confidence through sampling and aggregation.
Instead of relying on a single LLM output, the system:
- Performs repeated nugget extraction
- Clusters results across and within documents
- Highlights confidence to support transparent, reproducible evidence exploration
🩺 Real-world evaluation in urology
Together with our clinical partners at
🏥 Caritas-Krankenhaus St. Josef and
🏥 Barmherzige Brüder Klinikum Straubing,
we evaluated MedNuggetizer on the debated topic of antibiotic prophylaxis before prostate biopsy.
Using four major guidelines (EAU & AWMF, 2024–2025) and ten recent PubMed-indexed studies (systematic reviews and randomized controlled trials), two urologists manually assessed:
- 155 clusters (coherence/consistency)
- 406 information nuggets (query relevance)
Results show:
✅ High relevance of extracted nuggets
✅ Meaningful clustering into multiple information layers (context, current evidence, recommendations, limitations)
⚠️ Areas for improvement in the nuggets (undefined abbreviations, contextualization, partial cluster overlap)
👥 Team effort 🧑💻🤝🩺
Technical lead: Gregor Donabauer, Samy Ateia, and Udo Kruschwitz (Information Science Regensburg)
Clinical application: our colleagues in urology in Caritas-Krankenhaus St. Josef and Barmherzige Brüder Klinikum Straubing: Maximilian Burger, Matthias May, Christian Gilfrich, Maximilian Haas, Julio Rubén Rodas Garzaro, and Christoph Eckl
To support reproducibility, you can find the app’s source code and evaluation data on GitHub.
📄 Pre-print: https://arxiv.org/abs/2512.15384 (externer Link, öffnet neues Fenster)
💻 GitHub repository: https://github.com/SamyAteia/mednuggetizer-ecir2026 (externer Link, öffnet neues Fenster)
We are excited to demo MedNuggetizer at ECIR 2026 in Delft! Come chat with us about how confidence-based nugget extraction can strengthen trustworthy evidence synthesis in medicine. 😊🇳🇱
#ProfessionalSearch #MedicalSearch
#Urology #ProstateBiopsy
#InformationExtraction #InformationNuggets
#LargeLanguageModels #LLMs #ClinicalAI
#AIResearch #PhDResearch
#MedicalInformatics #HealthTech
#Reproducibility #OpenScience
#InformationRetrieval #IR
#NaturalLanguageProcessing #NLP
#EuropeanConferenceOnInformationRetrieval #ECIR #ECIR26 #ECIR2026
#ResearchSuccess #ResearchPaperAccepted
#InformationScienceRegensburg #StayInformed
