SketchHealer: A Graph-to-Sequence Network for Recreating Partial Human Sketches
Guoyao Su, Yonggang Qi, Kaiyue Pang, Jie Yang, Yi-Zhe Song
Abstract
To perceive and create a whole from parts is a prime trait of the human visual system. \nIn this paper, we teach machines to perform a similar task by recreating a vectorised \nhuman sketch from its incomplete parts. This is fundamentally different to prior work on \nimage completion (i) sketches exhibit a severe lack of visual cue and are of a sequential \nnature, and more importantly (ii) we ask for an agent that does not just fill in a missing \npart, but to recreate a novel sketch that closely resembles the partial input from scratch. \nCentral to our contribution is a graph model that encodes both the visual and structural \nfeatures over multiple categories. A novel sketch graph construction module is proposed \nthat leverages the sequential nature of sketches to associate key parts centred around \nstroke junctions. The intuition is then that message passing within the said graph will \nnaturally provide the healing power when it comes to missing parts (nodes). Finally, an \noff-the-shelf LSTM-based decoder is employed to decode sketches in a vectorised fashion. \nBoth qualitative and quantitative results show that the proposed model significantly \noutperforms state-of-the-art alternatives.