Anthropomorphism, False Beliefs, and Conversational <scp>AIs</scp> : How Chatbots Undermine Users' Autonomy

Abstract

ABSTRACT Conversational AIs (CAIs) are autonomous systems capable of engaging in natural language interactions with users. Recent advancements have enabled CAIs to engage in conversations with users that are virtually indistinguishable from human interactions. The proliferation of advanced CAIs creates a significant risk of users misattributing human‐like traits to CAIs. Such false beliefs can occur when the CAI's nature is not disclosed and users mistakenly believe they are interacting with a human, or even if the CAI is disclosed, through subconscious anthropomorphism. Existing literature on anthropomorphism and AI addresses the instrumental harms associated with anthropomorphism. I argue that anthropomorphizing CAIs might be bad in itself as it undermines user autonomy. My analysis focuses on how anthropomorphic false beliefs lead users to misapply behavioural norms, thereby affecting their autonomy. My argument will proceed as follows. I discuss the prevalence of CAIs, and establish the empirical claim that users are likely to form anthropomorphic false beliefs about CAIs. I explore existing accounts of how some false beliefs undermine autonomy and outline a plausible characterization to identify which false beliefs do so. I apply this characterization to CAIs and argue that misattributing human‐like capabilities to CAIs is the kind of false belief that undermines autonomy.

Topics & Concepts