TY  - JOUR
AU  - Berman, Eliza
AU  - Sundberg Malek, Holly
AU  - Bitzer, Michael
AU  - Malek, Nisar
AU  - Eickhoff, Carsten
PY  - 2025
DA  - 2025/3/5
TI  - Retrieval Augmented Therapy Suggestion for Molecular Tumor Boards: Algorithmic Development and Validation Study
JO  - J Med Internet Res
SP  - e64364
VL  - 27
KW  - large language models
KW  - retrieval augmented generation
KW  - LLaMA
KW  - precision oncology
KW  - molecular tumor board
KW  - molecular tumor
KW  - LLMs
KW  - augmented therapy
KW  - MTB
KW  - oncology
KW  - tumor
KW  - clinical trials
KW  - patient care
KW  - treatment
KW  - evidence-based
KW  - accessibility to care
AB  - Background: Molecular tumor boards (MTBs) require intensive manual investigation to generate optimal treatment recommendations for patients. Large language models (LLMs) can catalyze MTB recommendations, decrease human error, improve accessibility to care, and enhance the efficiency of precision oncology. Objective: In this study, we aimed to investigate the efficacy of LLM-generated treatments for MTB patients. We specifically investigate the LLMs’ ability to generate evidence-based treatment recommendations using PubMed references. Methods: We built a retrieval augmented generation pipeline using PubMed data. We prompted the resulting LLM to generate treatment recommendations with PubMed references using a test set of patients from an MTB conference at a large comprehensive cancer center at a tertiary care institution. Members of the MTB manually assessed the relevancy and correctness of the generated responses. Results: A total of 75% of the referenced articles were properly cited from PubMed, while 17% of the referenced articles were hallucinations, and the remaining were not properly cited from PubMed. Clinician-generated LLM queries achieved higher accuracy through clinician evaluation than automated queries, with clinicians labeling 25% of LLM responses as equal to their recommendations and 37.5% as alternative plausible treatments. Conclusions: This study demonstrates how retrieval augmented generation–enhanced LLMs can be a powerful tool in accelerating MTB conferences, as LLMs are sometimes capable of achieving clinician-equal treatment recommendations. However, further investigation is required to achieve stable results with zero hallucinations. LLMs signify a scalable solution to the time-intensive process of MTB investigations. However, LLM performance demonstrates that they must be used with heavy clinician supervision, and cannot yet fully automate the MTB pipeline. 
SN  - 1438-8871
UR  - https://www.jmir.org/2025/1/e64364
UR  - https://doi.org/10.2196/64364
UR  - http://www.ncbi.nlm.nih.gov/pubmed/40053768
DO  - 10.2196/64364
ID  - info:doi/10.2196/64364
ER  -