Utilizing Artificial Intelligence for Text Classification in Communication Sciences
Sadettin Demirel, Neslihan Bulur, Zindan Çakıcı
Abstract
This study delves into the evaluation of ChatGPT's effectiveness in sentiment detection and text classification tasks specifically on Turkish texts, a domain that has been relatively underexplored in existing literature predominantly focused on English texts. Leveraging datasets comprising manually labeled YouTube comments and news tweets categorized into sentiment classes and thematic topics, the authors rigorously assess the performance of ChatGPT-3.5 and ChatGPT-4 using accuracy and F1 performance metrics. These findings unveil insights into ChatGPT's proficiency in classifying Turkish textual content, illuminating its alignment with human-labeled classifications. This research not only contributes to expanding the scope of AI research beyond English language but also underscores the significance of language diversity in evaluating and refining AI models' performance for broader applicability in the research practices of social and communication sciences.