GPT-4 as a biomedical simulator

• Published in: Computers in biology and medicine 2024-08, Vol.178, p.108796, Article 108796
• Main Authors: Schaefer, Moritz, Reichl, Stephan, ter Horst, Rob, Nicolas, Adele M., Krausgruber, Thomas, Piras, Francesco, Stepper, Peter, Bock, Christoph, Samwald, Matthias
• Format: Article
• Language: English
• Subjects: Application programming interface; Artificial intelligence; Biological activity; Biomedical simulation; Cancer; Computational biology; Correlation coefficient; Correlation coefficients; Domains; Experiments; First principles; GPT-4; Immunology; Inference; Language; Large language models; Mathematical models; Medical research; Methods; Predictions; Qualitative analysis; Sepsis; Simulation; Simulators; Software; Survival
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.108796

Computational simulation of biological processes can be a valuable tool for accelerating biomedical research, but usually requires extensive domain knowledge and manual adaptation. Large language models (LLMs) such as GPT-4 have proven surprisingly successful for a wide range of tasks. This study provides proof-of-concept for the use of GPT-4 as a versatile simulator of biological systems. We introduce SimulateGPT, a proof-of-concept for knowledge-driven simulation across levels of biological organization through structured prompting of GPT-4. We benchmarked our approach against direct GPT-4 inference in blinded qualitative evaluations by domain experts in four scenarios and in two quantitative scenarios with experimental ground truth. The qualitative scenarios included mouse experiments with known outcomes and treatment decision support in sepsis. The quantitative scenarios included prediction of gene essentiality in cancer cells and progression-free survival in cancer patients. In qualitative experiments, biomedical scientists rated SimulateGPT's predictions favorably over direct GPT-4 inference. In quantitative experiments, SimulateGPT substantially improved classification accuracy for predicting the essentiality of individual genes and increased correlation coefficients and precision in the regression task of predicting progression-free survival. This proof-of-concept study suggests that LLMs may enable a new class of biomedical simulators. Such text-based simulations appear well suited for modeling and understanding complex living systems that are difficult to describe with physics-based first-principles simulations, but for which extensive knowledge is available as written text. Finally, we propose several directions for further development of LLM-based biomedical simulators, including augmentation through web search retrieval, integrated mathematical modeling, and fine-tuning on experimental data. [Display omitted] •Proof-of-concept: large language models (LLMs) as biomedical simulators using GPT-4.•GPT-4 based simulator outperforms direct inference in qualitative expert assessments.•Achieved enhanced classification accuracy for gene essentiality in cancer cells.•Improved regression capabilities for progression-free survival prediction in cancer.•Roadmap for development and application of LLMs as universal biomedical simulators.