Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text
Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, Yejin Choi
2022Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)86 citationsDOIOpen Access PDF
Abstract
Modern neural language models can produce remarkably fluent and grammatical text. So much, in fact, that recent work by Clark et al. (2021) has reported that conventional crowdsourcing can no longer reliably distinguish between machine-authored (GPT-3) and humanauthored writing. As errors in machine generations become ever subtler and harder to spot, it poses a new challenge to the research community for robust machine text evaluation.
Topics & Concepts
Computer scienceCrowdsourcingArtificial intelligenceAnnotationNatural language processingMachine learningDecoding methodsLanguage modelInformation retrievalWorld Wide WebTelecommunicationsTopic ModelingNatural Language Processing TechniquesExplainable Artificial Intelligence (XAI)