MAR-Net: Motion-Assisted Reconstruction Network for Unsupervised Video Summarization
Yunzuo Zhang, Yameng Liu, Weili Kang, Yuxin Zheng
Abstract
Video summarization targets to extract the most important segments from a video by spatiotemporal analysis. Previous methods primarily learn content within videos based on appearance information, with a rare discussion on the effective utilization of motion information, which is equally essential to video understanding. In this letter, we expound upon a Motion-Assisted Reconstruction Network (MAR-Net), which synergistically models appearance and motion information within videos for unsupervised video summarization without any manual annotations. MAR-Net notably comprises a Bidirectional Modality Encoder (BiME) and a Video Context Navigator (VCN). By integrating uni-modal and cross-modal feature aggregation into a unified module, BiME allows for exploring sophisticated dependency relationships among features through a bidirectional attention mechanism. VCN can promote the semantic consistency between the cross-modal contexts and the input video by a consistency loss term, alleviating the noisy impact within the motion stream. Empirical results conducted on benchmark datasets demonstrate that MAR-Net outperforms other state-of-the-art methods.