Comparative Analysis of Transformer-based Large Language Models (LLMs) for Text Summarization

Abstract

Text summarization is a key application of Natural Language Processing (NLP) that involves condensing a piece of text to its essential meaning or main points. The advancements of Large Language Models (LLMs) have significantly improved text summarization. These LLMs, characterized by their extensive training on vast amounts of textual data, have emerged as a promising avenue for enhancing summarization techniques. This research paper examines the transformative potential of LLMs in revolutionizing the landscape of text summarization. This research work analyzes the performance of various LLMs, including BART (Bidirectional and Auto-Regressive Transformer), T5(Text-To-Text Transfer Transformer), PEGASUS (Pre-training with Extracted Gap-sentences for Abstractive Summarization), GPT (Generative Pre-Trained Transformers), and BERTSum (Bidirectional Encoder Representations from Transformers for Summarization). Further this research work discusses application on CNN Daily Mail dataset, which has been used widely for text summarization benchmark. Through the lens of comprehensive evaluation metrics such as ROUGE scores and key characteristics, the research aims to offer a nuanced understanding of how these LLMs fare in the summarization task. The insights extracted from this analysis hold significant promise for both researchers and practitioners in the NLP domain, offering valuable guidance on harnessing the power of LLMs for real-world applications. As per observations from this research work, BART, PEGASUS, and T5 are better choices for text summarization with a ROGUE-L score 40.90, 41.30 and 40.69 respectively. These model works very well for lengthy, scientific articles where abstractive summaries are desired among all of them. BERTsum works well for extractive summarization techniques. Although GPT is the fastest, having a Rogue-L score of 26.58 is not much effective for generating summary. Furthermore, this investigation lays the groundwork for the development of sophisticated Generative AI applications, poised to address a diverse array of business challenges across industries.

Topics & Concepts