A multimodal deep learning framework for real-time defect recognition in industrial components using visual, acoustic and vibration signals
Milad Rahmati, Nima Rahmati
Abstract
Purpose In modern manufacturing, early detection of defects in industrial components is critical for ensuring product quality, operational safety and production efficiency. Traditional inspection techniques based solely on visual or acoustic features often fall short in detecting subtle or internal faults, particularly in high-speed production environments. This paper presents a novel multimodal deep learning framework that integrates visual, acoustic and vibration signals to enable real-time, robust defect recognition in industrial components. Design/methodology/approach By fusing features from convolutional neural networks (CNNs) for image data, recurrent neural networks (RNNs) for acoustic sequences and signal transformers for vibration time series, our architecture captures cross-modal correlations and temporal dependencies that are often overlooked in unimodal systems. The framework is trained and evaluated on a custom-built dataset comprising synchronized visual, audio and accelerometer recordings from industrial processes, encompassing both surface and internal defect types. Findings Experimental results on a simulated dataset demonstrate that the proposed model significantly outperforms unimodal baselines and conventional machine learning approaches, achieving up to 94.7% classification accuracy with minimal latency, suggesting potential suitability for deployment on edge devices, though real-world validation is needed to account for environmental complexities like noise and sensor drift. Furthermore, interpretability analyses using Grad-CAM and SHAP reveal the contribution of each modality toward the final decision, enhancing model transparency. Originality/value The findings contribute to advancing intelligent quality control systems and align with the growing demand for smart, resilient manufacturing.