V4D: 4D Covolutional Neural Networks for Video-level Representations Learning
Shiwen Zhang, Sheng Guo, Weilin Huang, Matthew R. Scott, Limin Wang
2020International Conference on Learning Representations11 citations
Abstract
Most existing 3D CNN structures for video representation learning are clip-based methods, and do not consider video-level temporal evolution of spatio-temporal features. In this paper, we propose Video-level 4D Convolutional Neural Networks, namely V4D, to model the evolution of long-range spatio-temporal representation with 4D convolutions, as well as preserving 3D spatio-temporal representations with residual connections. We further introduce the training and inference methods for the proposed V4D. Extensive experiments are conducted on three video recognition benchmarks, where V4D achieves excellent results, surpassing recent 3D CNNs by a large margin.
Topics & Concepts
Computer scienceMargin (machine learning)Convolutional neural networkArtificial intelligenceRepresentation (politics)ResidualInferencePattern recognition (psychology)Feature learningDeep learningMachine learningAlgorithmPolitical sciencePoliticsLawHuman Pose and Action RecognitionAdvanced Vision and ImagingVideo Surveillance and Tracking Methods