Recent Progress in Automated Code Generation from GUI Images Using Machine Learning Techniques
Daniel Baulé, Christiane Gresse von Wangenheim, Aldo von Wangenheim, Jean Carlo Rossa Hauck
Abstract
The manual transformation of a user interface design into code is a costly and time-consuming process. A solution can be the automation of the generation of code based on sketches or GUI design images. Recently, Machine Learning approaches have shown promising results in detecting GUI elements for such automation. Thus, to provide an overview of existing approaches, we performed a systematic mapping study. As a result, we identified and compared 20 approaches, that demonstrate good performance results being considered useful. These results can be used by researchers and practitioners in order to improve the efficiency of the GUI design process as well as continue to evolve and improve approaches for its support.