Evaluating Lightweight Transformers With Local Explainability for Android Malware Detection

Abstract

Mobile phones have evolved into powerful handheld computers, fostering a vast application ecosystem but also increasing security and privacy risks. Traditional deep learning-based Android malware detection, reliant on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), struggles to capture long-range dependencies, which are critical for identifying complex malware patterns. Transformers, with their self-attention mechanism, offer a promising alternative but are often computationally intensive for mobile deployment. To tackle this gap, this study assesses ten models—five customized architectures and five fine-tuned lightweight transformers (DistilBERT, CodeBERT, TinyBERT, MobileBERT, ALBERT), using a real-world dataset of 100K Android applications from Koodous, with API calls and permissions as features. Fine-tuned DistilBERT achieves 91.6% accuracy and 96.5% AUC, outperforming custom variants (up to 90.5% accuracy), highlighting transfer learning’s advantage. It remains competitive compared to AutoGluon leaderboard models (90–92% accuracy).With an average of 4.46±0.43 ms inference time and a 275 MB memory footprint, it balances efficiency and accuracy better than heavier transformers. Local Interpretable Model-Agnostic Explanations (LIME) are further integrated, with explanations aligning closely with VirusTotal’s malware descriptions. The findings demonstrate the viability of lightweight transformers for real-time Android malware detection, balancing accuracy, efficiency, and interpretability.

Topics & Concepts