Equivariant and Invariant Grounding for Video Question Answering
Yicong Li, Xiang Wang, Junbin Xiao, Tat‐Seng Chua
2022Proceedings of the 30th ACM International Conference on Multimedia30 citationsDOIOpen Access PDF
Abstract
Video Question Answering (VideoQA) is the task of answering the natural language questions about a video. Producing an answer requires understanding the interplay across visual scenes in video and linguistic semantics in question. However, most leading VideoQA models work as black boxes, which make the visual-linguistic alignment behind the answering process obscure. Such black-box nature calls for visual explainability that reveals "What part of the video should the model look at to answer the question?". Only a few works present the visual explanations in a post-hoc fashion, which emulates the target model's answering process via an additional method.
Topics & Concepts
Question answeringComputer scienceNatural languageSemantics (computer science)Process (computing)Artificial intelligenceTask (project management)Natural language processingProgramming languageEconomicsManagementMultimodal Machine Learning ApplicationsDomain Adaptation and Few-Shot LearningHuman Pose and Action Recognition