Comparative Analysis of ChatGPT-4 and LLaMA: Performance Evaluation on Text Summarization, Data Analysis, and Question Answering
Srinivasa Rao Bogireddy, Nagaraju Dasari
Abstract
ChatGPT has demonstrated the strong performance of large language models (LLMs) in natural language tasks since its November 2022 rollout by using vast datasets with trillions of parameters for training. This paper compares ChatGPT-4 and LLaMA on three NLP topics: text summarization, data analysis, and question answering using CNN/DailyMail, Airbnb Activity, and SQuAD as some of the datasets used. ChatGPT-4 outperforms LLaMA in all aspects, giving better accuracy, coherence, and relevance results, albeit with slightly higher computation costs. ChatGPT-4 performs well regarding text summarization, resulting from high BLEU scores and human evaluations for coherence, relevance, and readability. Data analyzed by ChatGPT-4 will give a more accurate insight into this topic and provide higher-quality visualizations and more precise outputs. For question answering it ranks above LLaMA in terms of precision, recall, F1 scores, correctness, and relevancy according to human ratings.