Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, Zhe Gan
Topics & Concepts
Computer scienceHuman–computer interactionGrounded theoryUser interfaceMultimediaProgramming languageQualitative researchSociologySocial scienceSpeech and dialogue systemsICT in Developing CommunitiesMobile and Web Applications