A Comparison of Principal Component Analysis, Maximum Likelihood and the Principal Axis in Factor Analysis

Abstract

This study aims to draw attention to the best extraction technique that may be considered when using the three of the most popular methods for choosing the number of factors/components: Principal Component Analysis (PCA), Maximum Likelihood Estimate (MLE) and Principal Axis Factor Analysis (PAFA), and compare their performance in terms of reliability and accuracy. To achieve this study objective, the analysis of the three methods was subjected to various research contexts. A Monte Carlo method was used to simulate data. It generates a number of datasets for the five statistical distribution considered in this study: The Normal, Uniform, Exponential, Laplace and Gamma distributions. The level of improvement in the estimates was related to the proportion of observed variables and the sum of the square loadings of the factors/components within the dataset and across the studied distributions. Different combinations of sample size and number of variables over the distributions were used to perform the analysis on the three analyzed methods. The generated datasets consist of 8 and 20 variables and 20 and 500 number of observations for each variable. 8 and 20 variables were chosen to represent small and large variables respectively. Also 20 and 500 sample sizes were chosen to represent also the small and large sample sizes respectively. The result of analysis, from applying the procedures on the simulated data set, confirm that PC analysis is overall most suitable, although the loadings from PCA and PAFA are rather similar and do not differ significantly, though the principal component method yielded factors that load more heavily on the variables which the factors hypothetically represent. Considering the above conclusions, it would be natural to recommend the use of PCA over other extraction methods even though PAF is somehow similar to its methods.

Topics & Concepts