Database Systems in the Big Data Era: Architectures, Performance, and Open Challenges
Ηλίας Δρίτσας, Μαρία Τρίγκα
Abstract
The advent of Big Data has fundamentally transformed the field of database management systems (DBMS), necessitating the development of innovative paradigms, architectures, and technologies to address unprecedented challenges. Traditional systems, despite their historical dominance, falter under the high velocity, massive volume, and diverse variety of Big Data. These limitations have catalyzed the emergence of alternative solutions such as No (structured query language) SQL, NewSQL, and cloud-native databases, each offering unique approaches to scalability, flexibility, and performance optimization. This survey provides a comprehensive and systematic overview of the evolving database ecosystem in the Big Data era. It delves into the historical progression from traditional relational database management systems (RDBMS) to modern paradigms, emphasizing the motivations, trade-offs, and innovations inherent in these transformations. The classification of databases based on data models, deployment strategies, scalability mechanisms, and consistency models is explored in depth, providing a structured framework for understanding their diverse capabilities. Furthermore, critical performance characteristics, including throughput, latency, fault tolerance, and cost efficiency, are analyzed to assess their effectiveness in real-world applications. By highlighting persistent challenges such as data heterogeneity, security, and interoperability, this survey outlines key research directions, fostering a holistic understanding of the domain and inspiring future advancements in database technologies.