Cost-Efficient Feature Selection for Horizontal Federated Learning
Sourasekhar Banerjee, Devvjiit Bhuyan, Erik Elmroth, Monowar Bhuyan
Abstract
Horizontal federated learning (HFL) exhibits substantial similarities in feature space across distinct clients. However, not all features contribute significantly to the training of the global model. Moreover, the curse of dimensionality delays the training. Therefore, reducing irrelevant and redundant features from the feature space makes training faster and inexpensive. This work aims to identify the common feature subset from the clients in federated settings. We introduce a hybrid approach called Fed-MOFS,<xref ref-type="fn" rid="fn1"><sup>1</sup></xref><fn id="fn1"><label><sup>1</sup></label> This manuscript is an extension of Banerjee et al. <xref ref-type="bibr" rid="ref1">[1]</xref>. </fn> utilizing mutual information (MI) and clustering for local FS at each client. Unlike the Fed-FiS, which uses a scoring function for global feature ranking, Fed-MOFS employs multiobjective optimization to prioritize features based on their higher relevance and lower redundancy. This article compares the performance of Fed-MOFS<xref ref-type="fn" rid="fn2"><sup>2</sup></xref><fn id="fn2"><label><sup>2</sup></label> We share our code, data, and supplementary copy through <uri>https://github.com/DevBhuyan/Horz-FL/blob/main/README.md</uri>. </fn> with conventional and federated FS methods. Moreover, we tested the scalability, stability, and efficacy of both Fed-FiS and Fed-MOFS across diverse datasets. We also assessed how FS influenced model convergence and explored its impact in scenarios with data heterogeneity. Our results show that Fed-MOFS enhances global model performance with a 50% reduction in feature space and is at least twice as fast as the FSHFL method. The computational complexity for both approaches is O(<inline-formula><tex-math notation="LaTeX">$d^{2}$</tex-math></inline-formula>), which is lower than the state of the art.