Large language models: an overview of foundational architectures, recent trends, and a new taxonomy
Ibomoiye Domor Mienye, Nobert Jere, George Obaido, Oyindamola Omolara Ogunruku, Ebenezer Esenogho, Cameron Modisane
Abstract
Abstract Since the introduction of foundational models such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformers (GPT), there has been rapid evolution in both the scale and application of large language models (LLMs). This review provides a concise overview of LLMs, their architecture, training methodologies, and recent innovative applications, focusing on notable models such as the GPT series, BERT, Pathways Language Model (PaLM), and Large Language Model Meta AI (LLaMA), and recently the DeepSeek-R1 model. Additionally, this paper presents a taxonomy for categorizing LLMs based on three critical dimensions: scalability, application domains, and ethical considerations. This taxonomy aims to enable researchers and practitioners to better understand these models in terms of their potential and limitations. Lastly, by reviewing contributions from numerous publications, this study identifies emerging trends, gaps, and opportunities in LLM research, providing a structured guide for future investigations.