How effective is AI augmentation in human–AI collaboration? Evidence from a field experiment
Chengcheng Liao, Xin Wen, Shan Li, Peiyuan Du
Abstract
Purpose Companies increasingly leverage artificial intelligence (AI) to enhance human performance, particularly in e-commerce. However, the effectiveness of AI augmentation remains controversial. This study investigates whether, how and why AI enhances human agents’ sales through a randomized field experiment. Design/methodology/approach This study conducts a two-by-two factorial randomized field experiment ( N = 1,090) to investigate the effects of AI augmentation on sales. The experiment compares sales outcomes handled solely by human agents with those augmented by AI, while also examining the moderating effect of agents’ experience levels and the underlying mechanisms behind agents’ responses. Findings The results reveal that AI augmentation leads to a significant 5.46% increase in sales. Notably, the impact of AI augmentation varies based on agents’ experience levels, with inexperienced agents benefiting nearly six times more than their experienced counterparts. Mediation analysis shows that AI augmentation improves response timeliness, accuracy and sentiment, thereby boosting sales. Originality/value This study highlights the role of AI augmentation in human–AI collaboration, demonstrates the varying impacts of AI augmentation based on agents’ experience levels and offers insights for organizations on how to regulate AI augmentation to enhance agent responses and drive sales.