Multimodality Sentiment Analysis in Social Internet of Things Based on Hierarchical Attentions and CSAT-TCN With MBM Network
Guorong Xiao, Geng Tu, Lin Zheng, Teng Zhou, Xin Li, Syed Hassan Ahmed, Dazhi Jiang
Abstract
Multimodality sentiment analysis in the social Internet of Things is a developing field, which is basic to empathetic mechanisms, affective computing, and artificial intelligence. Current works in this domain do not explicitly consider the influence of contextual information fusion based on correlation coefficient and memory network with branch structure for sentiment analysis. Unlike present works, this article presents a hierarchical self-attention fusion (H-SATF) model for capturing contextual information better among utterances, a contextual self-attention temporal convolutional network (CSAT-TCN) for sentiment recognition in the social Internet of Things, and a multibranch memory (MBM) network that stores self-speaker and interspeaker sentimental states into global memories. For MOSI data sets, the hybrid H-SATF-CSAT-TCN-MBM model outperforms the state-of-the-art networks and shows 0.31%-9.93% improvement.