While a picture might convey a lot, it still falls short compared to what BiomedGPT can achieve.
Recently highlighted in the journal Nature Medicine, BiomedGPT represents a novel type of artificial intelligence (AI) aimed at assisting with various medical and scientific endeavors. The study, developed in partnership with several institutions, introduces what is termed “the first open-source and lightweight vision-language foundation model, designed as a generalist capable of performing various biomedical tasks.”
“This initiative merges two forms of AI into a decision support system for healthcare providers,” notes Lichao Sun, who is an assistant professor of computer science and engineering at Lehigh University and a principal author of the research. “One component of the system is programmed to interpret biomedical images, while the other focuses on understanding and evaluating biomedical texts. By merging these capabilities, the model can address numerous biomedical challenges, drawing insights from collections of biomedical images as well as from the analysis and integration of scientific and medical literature.”
’16 top results’ for healthcare professionals and patients
The groundbreaking feature mentioned in the August 7 Nature Medicine article, “A generalist vision-language foundation model for diverse biomedical tasks,” is that this AI does not require specific training for every individual task. Traditionally, AI models are developed for specific roles, such as identifying tumors in X-rays or synthesizing medical papers. In contrast, this novel model can manage a variety of tasks with the same foundational technology. This adaptability characterizes it as a “generalist” model and makes it an invaluable asset for healthcare providers.
“BiomedGPT is built on foundation models, which are an innovative advancement in AI,” explains Sun. “Foundation models are extensive, pre-trained AI systems that can be fine-tuned for a range of tasks with minimal extra training requirements. The generalist model discussed in the article has undergone extensive training on large datasets comprising both images and text in the biomedical field, enabling effective performance across various applications.”
“After evaluating 25 datasets spanning 9 biomedical tasks and various modalities,” states Kai Zhang, a Lehigh PhD student advised by Sun, who is the primary author of the Nature article, “BiomedGPT achieved 16 leading-edge results. Evaluations by humans of BiomedGPT on three radiology tasks highlighted the model’s strong predictive capabilities.”
Zhang expresses pride in the open-source codebase being available for other researchers to utilize as a foundation for further advancements and broader adoption.
The research team anticipates that BiomedGPT’s technology may eventually aid doctors in deciphering complex medical images, support researchers in analyzing scientific publications, or even facilitate drug discovery by predicting molecular behavior.
“The possible implications of this technology are substantial,” Zhang remarks, “as it could enhance various aspects of healthcare and research, making processes quicker and more precise. Our approach illustrates that effective training with diverse datasets can lead to more functional biomedical AI for enhancing diagnosis and workflow efficiency.”
A collaborative effort for clinical validation and beyond
A vital phase in this process involved confirming the model’s effectiveness and applicability in actual healthcare scenarios.
“Clinical evaluations consist of applying the AI model to genuine patient data to gauge its accuracy, dependability, and safety,” Sun describes. “Such assessments ensure the model’s effectiveness across different situations. The results from these tests were crucial in refining the model, showcasing its potential to enhance clinical decision-making and patient care.”
Massachusetts General Hospital (MGH), which is a foundational entity of the Mass General Brigham healthcare system and a teaching affiliate of Harvard Medical School, played a significant role in developing and validating the BiomedGPT model. Their involvement primarily centered on providing clinical insights and helping assess the model’s real-world effectiveness. For example, the model was evaluated by radiologists at MGH, where it outperformed in tasks such as visual question answering and generating radiology reports. This collaboration was essential to ensure the model’s accuracy and suitability for clinical application.
Additional contributors to BiomedGPT include researchers from the University of Georgia, Samsung Research America, the University of Pennsylvania, Stanford University, the University of Central Florida, UC-Santa Cruz, the University of Texas Health, Children’s Hospital of Philadelphia, and the Mayo Clinic.
“This research embodies a highly interdisciplinary and collaborative effort,” states Sun. “It integrates expertise from numerous fields, including computer science, medicine, radiology, and biomedical engineering. Every author brings specialized knowledge necessary for the model’s development, evaluation, and validation across multiple biomedical tasks. Large-scale endeavors like this rely heavily on having access to diverse datasets, computational resources, and skilled individuals in algorithm creation, model training, assessing, and applying the model in real-world contexts, alongside clinical tests and validations.”
“This was indeed a collective effort,” he continues. “Creating something capable of genuinely assisting the medical sector in enhancing patient outcomes across a broad spectrum of issues is exceedingly complex. With such intricacy, collaboration is essential for making a meaningful impact through the application of scientific and engineering principles.”
