Enhancing pine wilt disease detection with synthetic data and external attention-based transformers

Abstract

The catastrophic effects of Pine Wilt Disease (PWD) and the lack of a defined cure make it a danger to the world’s forests. The early detection of PWD becomes essential when establishing efficient approaches to mitigation. However, collecting data, labelling, and proposing a system for PWD detection is a challenge. This work presents an innovative PWD system and uses synthetic data sets to overcome these challenges. In particular, it emphasises two contributions: (1) the implementation of a multi-head external attention mechanism focused on improving computational effectiveness and model performance, and (2) the integration of synthetic data to address the problem of data scarcity in PWD detection. To solve the high computational cost of the self-attention function from the original transformer, we introduce external attention, which enables two linear transformations to take fewer amounts of parameters. Whereas the original self-attention is designed to capture relations inside a single sample, the external attention seeks inter-sample connections through memory modules. It is also computationally efficient compared to the wider context information, and it may consume as much computational power as the 1 by 1 convolution. Our external attention mechanism is multi-head attention, which enables the model to pay close attention to multiple features in the data and find different types of relationships. The proposed methodology achieves high performance across multiple metrics for both individual and ensemble datasets. For individual datasets, the model achieves accuracy (90%–98%), Global Detection Rate (GDR) (90%–98%), Matthews correlation coefficient (MCC) (0.80–0.96) and Kappa (0.80–0.96). For ensemble datasets, the model achieves accuracy (94%–96%), GDR (94%–96%), MCC (0.88 - 0.92) and Kappa (0.88 - 0.92). These results confirm that the presented technique is valid and effective in the early-stage detection of PWD in trees, and can be implemented in forest management.

Topics & Concepts