RAFT: Recurrent All-Pairs Field Transforms for Optical Flow (Extended Abstract)
Zachary Teed, Jia Deng
Abstract
We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance on the KITTI and Sintel datasets. In addition, RAFT has strong cross-dataset generalization as well as high efficiency in inference time, training speed, and parameter count.
Topics & Concepts
RaftComputer sciencePixelOptical flowGeneralizationArtificial intelligenceField (mathematics)InferenceFlow (mathematics)AlgorithmImage (mathematics)MathematicsPhysicsGeometryPolymerMathematical analysisCopolymerPure mathematicsNuclear magnetic resonanceAdvanced Vision and ImagingRetinal Imaging and AnalysisAdvanced Image Processing Techniques