Research on parallelization of Apriori algorithm in association rule mining
Huanbin Wang, Yangjun Gao
Abstract
Aiming at the performance bottleneck of traditional Apriori algorithm when the data set is slightly large, this paper adopts the idea of parallelization and improves the Apriori algorithm based on MapReduce model. Firstly, the local frequent itemsets on each sub node in the cluster are calculated, then all the local frequent itemsets are merged into the global candidate itemsets, and finally, the frequent itemsets that meet the conditions are filtered according to the minimum support threshold. The advantage of the improved algorithm is that it only needs to scan the transaction database twice and calculate the frequent item set in parallel, which improves the efficiency of the algorithm.
Topics & Concepts
Apriori algorithmComputer scienceAssociation rule learningBottleneckData miningDatabase transactionA priori and a posterioriSet (abstract data type)Node (physics)AlgorithmCluster (spacecraft)Transaction dataDatabasePhilosophyStructural engineeringEngineeringEpistemologyProgramming languageEmbedded systemData Mining Algorithms and ApplicationsImbalanced Data Classification TechniquesBig Data and Business Intelligence