Computational Approaches for Understanding, Generating, and Adapting User Interfaces
Yue Jiang, Yuwen Lu, Jeffrey Nichols, Wolfgang Stuerzlinger, Chun Yu, Christof Lutteroth, Yang Li, Ranjitha Kumar, Toby Jia-Jun Li
Abstract
Computational approaches for user interfaces have been used in adapting interfaces for different modalities, usage scenarios and device form factors, understanding screen semantics for accessibility, task-automation, information extraction, and in assisting interface design. Recent advances in machine learning (ML) have drawn considerable attention across HCI and related fields such as computer vision and natural language processing, leading to new ML-based user interface approaches. Similarly, significant progress has been made with more traditional optimization- and planning-based approaches to accommodate the need for adapting UIs for screens with different sizes, orientations and aspect ratios, and in emerging domains such as VR/AR and 3D interfaces. The proposed workshop seeks to bring together researchers interested in all kinds of computational approaches for user interfaces across different sectors as a community, including those who develop algorithms and models and those who build applications, to discuss common issues including the need for resources, opportunities for new applications, design implications for human-AI interaction in this domain, and practical challenges such as user privacy.