Motion Deblur by Learning Residual From Events
Kang Chen, Lei Yu
Abstract
Conventional cameras face challenges when capturing motion information during the exposure due to their physical design, rendering the motion deblurring task ill-posed. To this end, we propose a Two-stage Residual-based Motion Deblurring (TRMD) framework for an event camera, which converts a blurry image into a sequence of sharp images, leveraging the abundant motion features encoded in events. In the first stage, a residual estimation network is trained to estimate the residual sequence, which measures the intensity difference between the intermediate frame and other frames sampled during the exposure. In the subsequent stage, the previously estimated residuals are combined with the blurry image to reconstruct the deblurred sequence based on the physical model of motion blur. To facilitate the efficient integration of image and event modalities for residual estimation, we propose a cross-modal fusion module based on spatial-channel attention, aiming to fuse the complementary spatial-temporal features of two modalities. Extensive experiments demonstrate that our method outperforms current state-of-the-art approaches on the synthetic dataset GOPRO and produces superior visualization with less noise and artifacts on the real blur event dataset REBlur.