Multispectral image reconstruction from RGB image for maize growth status monitoring based on window-adaptive spatial-spectral attention transformer

Abstract

• An improved transformer model was proposed for MSI reconstruction from RGB images. • Reconstructed 10-band images achieved a correlation of 0.99 with original spectra. • PLSR model accurately predicted maize chlorophyll using reconstructed spectra. • RGB-based multispectral reconstruction enabled precise maize growth monitoring. Multispectral image analysis is an effective way to detect crop growth status. However, the complexity of manufacturing process and technology of multispectral image acquisition equipment make data acquisition expensive. Therefore, a method based on a window-adaptive spatial-spectral attention transformer is proposed to reconstruct multispectral images using RGB images of maize. First, RGB and hyperspectral images of the maize are obtained, and the reflectance data from classic and preferred band combinations are extracted from the hyperspectral image. Then, a transformer model is constructed to evaluate and compare the reconstruction efficacy of the 5-band and 10-band combinations across four attention modes: spatial, spectral, spatial-spectral, and window-adaptive spatial-spectral attention. The best-performing reconstruction results are selected and compared with the original data from three perspectives: image, spectrum, and model effect. The 10-band multispectral image reconstructed by the window-adaptive spatial-spectral attention mechanism is highly similar to the original image, with a reflectance correlation exceeding 0.99. Furthermore, its application in monitoring crop growth status (i.e., maize chlorophyll) yields results closely aligned with actual reflectance data: R C 2 is 0.76, R V 2 is 0.64, while RMSE C and RMSE V are 3.63 mg/L and 2.94 mg/L, respectively. To further explore the model performance, the new sensitive bands are selected to be reconstructed in the maize V7 stage. The results from the chlorophyll content prediction model are as: R C 2 is 0.64, R V 2 is 0.60, with RMSE C and RMSE V are 5.61 mg/L and 5.62 mg/L, respectively. Therefore, the window-adaptive spatial-spectral attention transformer can accurately reconstruct multispectral images and establish precise growth status monitoring models, providing technical support for low-cost field maize growth detection.

Topics & Concepts