How to Detect AI-Generated Texts?
Trung T. Nguyen, Amartya Hatua, Andrew H. Sung
Abstract
Recent advances in large language models (LLMs) have significantly improved the quality of synthetic text data. LLMs imitate human writing patterns to produce highly natural text, raising serious ethical, moral, legal, social, and economic concerns in various industries. To address these issues, we present two methods to distinguish Synthetically Generated Text (SGT) from Human-Written Text (HWT): machine learning and text similarity. Our methods include procedures for dataset creation, feature engineering, model training, and result analysis with feature importance analysis and explanation models. As part of our research, we created two datasets - a Wikipedia-based and a US Election 2024 news article-based dataset using ChatGPT, from which we obtained promising results of up to 99.xxx% accuracy. These datasets can be used as open-source datasets in future studies. We also identified a set of handcrafted features that can serve as the baseline feature set for future research.