The Evolution of AI: From Classical Machine Learning to Modern Large Language Models

Abstract

This paper provides a comprehensive review of the evolution of artificial intelligence from early symbolic, rule-based systems to modern large language models (LLMs) and retrieval-augmented generation (RAG) architectures. Early AI was dominated by symbolic reasoning and expert systems using hand-crafted rules, before a paradigm shift toward data-driven learning occurred with the advent of machine learning, notably the backpropagation algorithm that enabled training of multi-layer neural networks to learn complex tasks. This breakthrough ushered in the deep learning era: convolutional neural networks (CNNs) achieved landmark results in computer vision , recurrent neural networks (RNNs such as LSTM) excelled at sequential data processing, and deep reinforcement learning (RL) systems mastered challenging decision-making tasks. In natural language processing (NLP), distributed word embedding models like Word2Vec and GloVe replaced sparse representations, capturing semantic relationships in vector space. To handle rare and out-of-vocabulary words, subword tokenization techniques such as Byte Pair Encoding (BPE) and WordPiece were introduced. These advances culminated in the Transformer architecture, based on self-attention mechanisms, which now underpins most state-of-the-art LLMs. Transformer-based LLMs, pre-trained on large text corpora, demonstrate advanced capabilities in language generation and understanding. Beyond model architectures, the review highlights how LLMs are augmented with external knowledge via RAG. Retrieval-augmented generation combines an LLM with a vector database of text embeddings, allowing the model to retrieve relevant external information to ground its outputs. This approach significantly improves factual accuracy and relevance, mitigating issues like hallucination, though ensuring complete truthfulness remains challenging. Additional obstacles include the limited context windows of LLMs (on the order of a few thousand tokens) and the computational complexity and latency of inference. The review also examines societal implications of advanced AI, such as bias amplification from training data and other ethical considerations in deployment. Finally, it outlines future directions, including extended context lengths, improved interpretability and alignment, greater efficiency, and neuro-symbolic hybrid approaches, aiming to develop more robust and trustworthy AI systems.

Topics & Concepts