Deep Binocular Fixation Prediction Using a Hierarchical Multimodal Fusion Network
Wujie Zhou, Wenyu Liu, Jingsheng Lei, Ting Luo, Lu Yu
Abstract
RGB-D data are increasingly being used for myriad computer vision tasks. For such tasks, most methods simply concatenate or add feature vectors from RGB images and depth maps and allow the two modalities to complement each other mutually. However, such a fusion strategy results in inefficient and inadequate performance. In this study, we propose deep binocular fixation prediction based on a hierarchical multimodal fusion network that suitably combines RGB and depth maps. In the proposed method, a novel convolutional block attention module completely extracts image texture features and retains spatial information. In addition, a pyramid dilated-convolution module refines feature information, further improving the fusion of RGB and depth maps. Experimental results indicate that the proposed network achieves state-of-the-art performance on the NUS and NCTU data sets.