Aesthetics-Assisted Multi-task Learning with Attention for Image Memorability Prediction
Tong Zhu, Feng Zhu, Hancheng Zhu, Leida Li
Abstract
The rapid development of multimedia produces a huge amount of images every day. Some can be remembered by us for a long time, while most of them are quickly forgotten. This intrinsic property of an image is called memorability, which has wide applications in advertising, photography, education and so on. Therefore, image memorability has been drawing increasing interest recently. People's unique taste of beauty influences their memory of images, however, there are few works assisting memorability prediction by utilizing visual aesthetics assessment. In this paper, we propose a new aesthetics-assisted multi-task deep learning network for image memorability estimation. Visual attention mechanism is employed in our network to focus on the most useful context information. In order to capture the common features between image memorability and aesthetics, we use image memorability data and image aesthetics data to train the multi-task deep network with attention alternately. The experimental results demonstrate that the proposed method outperforms the state-of-the-arts.