A Smart Multimodal Biomedical Diagnosis Based on Patient's Medical Questions and Symptoms

Abstract

The exponential increase of health-related digital data has given machine learning algorithms a newfound ability to generate more meaningful insights. Information such as diagnosis, treatments, and prescriptions are all part of digital health data. In order to better care for their patients, healthcare providers provide crucial diagnostic services. Mistakes in diagnosis, however, lead to the patient receiving harmful treatment too soon or too late. In order to reduce the likelihood of clinical cognitive errors, computer-aided diagnosis techniques have been developed. The proposed approach utilizes a massive health-related data set, which is comprised of many unstructured patient questions written in various Arabic dialects, as well as symptoms reported by general practitioners (GPs). System components include a combination of machine learning models that have been trained using either patient symptoms or patient medical inquiries. Machine learning (ML) strategies, and variations of the multilayer perceptron (MLP) classifier have all been utilized in trials as feature representation techniques and ML classifiers. We also discuss the technical and analytical hurdles, as well as the most important new applications, that this research opens up. Possibilities in areas such as digital clinical trials, telehealth, pandemic surveillance, digital twins, and virtual health aides are discussed. We also provide an overview of the data, modeling, and privacy obstacles that must be surmounted before the healthcare industry can fully benefit from multimodal AI. With a classification accuracy of 94.9%, the combined results of the two modalities demonstrate promising prediction potential. The results show promise for using the algorithm to predict possible diagnoses of patient illnesses that depend on the given symptoms and queries, which can help doctors make more informed judgments.

Topics & Concepts