A Review of Deep Learning for Video Captioning
Moloud Abdar, Meenakshi Kollati, Swaraja Kuraparthi, Farhad Pourpanah, Daniel McDuff, Mohammad Ghavamzadeh, Shuicheng Yan, Abduallah Mohamed, Abbas Khosravi, Erik Cambria, Fatih Porikli
Abstract
Video captioning (VC) is a fast-moving, cross-disciplinary area of research that comprises contributions from domains such as computer vision, natural language processing, linguistics, and human-computer interaction. VC aims to understand a video and describe it through natural language descriptors. It plays a crucial role in various applications, from improving accessibility features such as low-vision navigation to advancing video question answering, video retrieval, and content generation. In this survey paper, we present a comprehensive review of deep learning-based VC methods. First, we provide an overview of VC, including the problem formulation, evaluation metrics, training losses, and attention-based architectures. Then, we categorize VC methods into several categories, including attention-based architectures graph networks, reinforcement learning, adversarial networks, and dense video captioning, and discuss each category in detail. In addition, we review existing data sets for VC methods and provide a discussion of research gaps and future research directions. We hope that this survey serves as a guide for researchers in relevant fields.