Video object detection from one single image through opto-electronic neural network
Chengyang Hu, Honghao Huang, Minghua Chen, Sigang Yang, Hongwei Chen
Abstract
An opto-electronic neural network is designed for video object detection from a long-exposure blurred image. This network combines an optical encoder, convolutional neural network decoder, and object detection module, which are jointly optimized end-to-end. The joint loss is adopted for updating the network according to the physical constraints of hardware via back-propagation. A high-speed refreshed spatial light modulator is used as the encoder part of the network to generate coded sub-images, and then, a single blurred image is obtained after a common camera. The rest of the network is used for video object detection. Both simulations and experiments demonstrate that our framework can successfully retrieve object labels and bounding boxes at different moments in the long exposure. To the best of our knowledge, this is the first work investigating video object detection from a single motion-degraded image.