User Interactions With a Municipality Chatbot—Lessons Learnt From Dialogue Analysis
Asbjørn Følstad, Nina Bjerkreim-Hanssen
Abstract
Chatbots are increasingly taken up by the public sector, as a means to efficient provision of information and services. However, there is a lack of knowledge on how users interact with such chatbots. To address this knowledge gap, we have conducted an analysis of user interactions with a chatbot for citizens of Norwegian municipalities. We analyzed a total of 2663 user-chatbot dialogues from six municipalities, using the framework of Følstad and Taylor. The analysis showed that most user input was characterized by brief messages and a utility-oriented dialogue style whereas chatbot responses were characterized by substantial response relevance (68% of chatbot responses categorized as relevant) and helpfulness (66% of dialogues categorized as help being offered and likely used). Furthermore, message brevity and a utility-oriented dialogue style was found to be positively associated with users receiving relevant chatbot responses and helpful dialogue outcomes. Variation in chatbot design, specifically how the chatbot was presented to users, was found to impact user message brevity and dialogue style, and, by extension, response relevance and dialogue outcome. On the basis of the findings, we summarize lessons learnt and suggest directions for future research.