Interactive Task Learning from GUI-Grounded Natural Language Instructions and Demonstrations
Toby Jia-Jun Li, Tom M. Mitchell, Brad A. Myers
Abstract
We show SUGILITE, an intelligent task automation agent that can learn new tasks and relevant associated concepts interactively from the user's natural language instructions and demonstrations, using the graphical user interfaces (GUIs) of third-party mobile apps. This system provides several interesting features:
Topics & Concepts
Computer scienceHuman–computer interactionTask (project management)Android (operating system)Graphical user interfaceNatural languageArchitectureUser interfaceNatural language understandingProgramming languageArtificial intelligenceEconomicsManagementVisual artsArtOperating systemSpeech and dialogue systemsTopic ModelingAI in Service Interactions