Parallelization of $Top_{k}$ Algorithm Through a New Hybrid Recommendation System for Big Data in Spark Cloud Computing Framework
Kaoutar El Handri, Abdellah Idrissi
Abstract
In the era of big data, parallel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Top_{k}$</tex-math></inline-formula> query processing under information retrieval has received increasing attention from both the industry and academia. This query handling allows users to retrieve the most useful data objects in a set of choices. This problem is compounded by the use of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Top_{k}$</tex-math></inline-formula> in cases of multiple dimensions and extensive data analytics. In this article, we provide a novel parallel algorithm in a distributed recommender system based on the Apache Spark platform. The purpose of this approach was to implement the multicriteria decision aiding support and dominating query approach run by using matrix factorization and singular value decomposition (SVD)-based model as a sophisticated machine learning technique. Simultaneously, applying the resilient distributed datasets paradigm in cloud computing, which presents a favorable environment for big data management. Extensive experimental results in terms of accuracy, and scalability indicated the new algorithm’s advantage compared to other <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Top_{k}$</tex-math></inline-formula> algorithms. Accordingly, our recommender system based on the conceived algorithm achieved high precision (62%–82%, depending on the data) to verify the profoundly positive effect of the use of the Spark framework and the SVD-based model while applying the commonly used evaluation metrics in the recommendation systems.