Long-term Leap Attention, Short-term Periodic Shift for Video Classification
Hao Zhang, Lechao Cheng, Yanbin Hao, Chong‐Wah Ngo
2022Proceedings of the 30th ACM International Conference on Multimedia12 citationsDOIOpen Access PDF
Abstract
Video transformer naturally incurs a heavier computation burden than a static vision transformer, as the former processes T times longer sequence than the latter under the current attention of quadratic complexity (T2N2). The existing works treat the temporal axis as a simple extension of spatial axes, focusing on shortening the spatio-temporal sequence by either generic pooling or local windowing without utilizing temporal redundancy.
Topics & Concepts
TransformerComputer scienceComputationAlgorithmQuadratic equationPoolingTheoretical computer scienceArtificial intelligenceMathematicsVoltageGeometryElectrical engineeringEngineeringHuman Pose and Action RecognitionAdvanced Vision and ImagingVideo Surveillance and Tracking Methods