Joint Reinforcement and Contrastive Learning for Unsupervised Video Summarization
Yunzuo Zhang, Yameng Liu, Pengfei Zhu, Weili Kang
Abstract
This letter presents a joint Reinforcement and Contrastive Learning framework termed RCL for unsupervised video summarization, aiming at addressing the existing two shortcomings: (i) poor feature representation, and (ii) inefficient context modeling capability. Concretely, the proposed framework consists of an Optimized Coding Module (OCM) and a Dissimilarity-Guided Attention Graph (DGAG). The OCM is grounded on Gate Recurrent Unit (GRU), which encodes the content within shots into concise representations. Different from the existing approaches, contrastive learning is introduced to promote discriminative and informative feature learning. Afterward, the DGAG adaptively performs feature aggregation by evaluating the semantic dissimilarity across shots to eliminate chaotic message passing for accurate context modeling. Finally, the procedure of importance score prediction is formulated as a node classification task, and these scores are utilized for a summary generation. Extensive experiments on the benchmark datasets demonstrate the superior performance of the proposed method.