Umeå University's logo

In the era of big data and federated learning, traditional feature selection methods show unacceptable performance for handling heterogeneity when deployed in federated environments. We propose Fed-FiS, an information-theoretic federated feature selection approach to overcome the problem occur due to heterogeneity. Fed-FiS estimates feature-feature mutual information (FFMI) and feature-class mutual information (FCMI) to generate a local feature subset in each user device. Based on federated values across features and classes obtained from each device, the central server ranks each feature and generates a global dominant feature subset. We show that our approach can find stable features subset collaboratively from all local devices. Extensive experiments based on multiple benchmark iid (independent and identically distributed) and non-iid datasets demonstrate that Fed-FiS significantly improves overall performance in comparison to the state-of-the-art methods. This is the first work on feature selection in a federated learning system to the best of our knowledge.

Federated Learning (FL) has evolved as a promising distributed learning paradigm in which data samples are disseminated over massively connected devices in an IID (Identical and Independent Distribution) or non-IID manner. FL follows a collaborative training approach where each device uses local training data to train local models, and the server generates a global model by combining the local model's parameters. However, FL is vulnerable to system heterogeneity when local devices have varying computational, storage, and communication capabilities over time. The presence of stragglers or low-performing devices in the learning process severely impacts the scalability of FL algorithms and significantly delays convergence. To mitigate this problem, we propose Fed-MOODS, a Multi-Objective Optimization-based Device Selection approach to reduce the effect of stragglers in the FL process. The primary criteria for optimization are to maximize: (i) the availability of the processing capacity of each device, (ii) the availability of the memory in devices, and (iii) the bandwidth capacity of the participating devices. The multi-objective optimization prioritizes devices from fast to slow. The approach involves faster devices in early global rounds and gradually incorporating slower devices from the Pareto fronts to improve the model's accuracy. The overall training time of Fed-MOODS is 1.8× and 1.48× faster than the baseline model (FedAvg) with random device selection for MNIST and FMNIST non-IID data, respectively. Fed-MOODS is extensively evaluated under multiple experimental settings, and the results show that Fed-MOODS has significantly improved model's convergence and performance. Fed-MOODS maintains fairness in the prioritized participation of devices and the model for both IID and non-IID settings.

Horizontal Federated Learning exhibits substantial similarities in feature space across distinct clients. However, not all features contribute significantly to the training of the global model. Moreover, the curse of dimensionality delays the training. Therefore, reducing irrelevant and redundant features from the feature space makes training faster and inexpensive. This work aims to identify the common feature subset from the clients in federated settings. We introduce a hybrid approach called Fed-MOFS 1 , utilizing Mutual Information and Clustering for local feature selection at each client. Unlike the Fed-FiS, which uses a scoring function for global feature ranking, Fed-MOFS employs multi-objective optimization to prioritize features based on their higher relevance and lower redundancy. This paper compares the performance of Fed-MOFS 2 with conventional and federated feature selection methods. Moreover, we tested the scalability, stability, and efficacy of both Fed-FiS and Fed-MOFS across diverse datasets. We also assessed how feature selection influenced model convergence and explored its impact in scenarios with data heterogeneity. Our results show that Fed-MOFS enhances global model performance with a 50% reduction in feature space and is at least twice as fast as the FSHFL method. The computational complexity for both approaches is O( d 2 ), which is lower than the state-of-the-art.

The key challenge of personalized federated learning (PerFL) is to capture the statistical heterogeneity properties of data with inexpensive communications and gain customized performance for participating devices. To address these, we introduced personalized federated learning in multi-tier architecture (PerMFL) to obtain optimized and personalized local models when there are known team structures across devices. We provide theoretical guarantees of PerMFL, which offers linear convergence rates for smooth strongly convex problems and sub-linear convergence rates for smooth non-convex problems. We conduct numerical experiments demonstrating the robust empirical performance of PerMFL, outperforming the state-of-the-art in multiple personalized federated learning tasks.

Lifelog images are very useful as memory cues for recalling past events. Estimating the level of event memory recall induced by a given lifelog image (event memorability) is useful for selecting images for cognitive interventions. Previous works for predicting event memorability follow a centralized model training paradigm that requires several users to share their lifelog images. This risks violating the privacy of individual lifeloggers. Alternatively, a personal model trained with a lifelogger’s own data guarantees privacy. However, it imposes significant effort on the lifelogger to provide a large enough sample of self-rated images to develop a well-performing model for event memorability. Therefore, we propose a clustered personalized federated learning setup, FedMEM, that avoids sharing raw images but still enables collaborative learning via model sharing. For an enhanced learning performance in the presence of data heterogeneity, FedMEM evaluates similarity among users to group them into clusters. We demonstrate that our approach furnishes high-performing personalized models compared to the state-of-the-art.

Sharing privacy-sensitive information poses a significant risk when private information is carelessly exposed to individuals or groups through various media. With minimal annotation and user preference, users can decide whether to share or keep an image based on its privacy score. This approach helps reduce the likelihood of privacy breaches and makes it easier to share images with friends or on social media platforms. Federated Learning (FL) allows collaborative learning of models without sharing personal data, which is crucial for sharing privacy-sensitive information. However, there are several challenges associated with learning from private data. This paper proposes two FL algorithms, namely Dynamic-Clustered-FedDC and Apriori-Clustered-FedDC, to address key problems, such as data scarcity, data heterogeneity, and model complexity, in order to ensure personalized image privacy. Both algorithms train personalized models for each annotator using clustered federated learning to address data heterogeneity. Additionally, both algorithms utilize daisy-chaining-based knowledge sharing between annotators to mitigate data scarcity issues during training. In addition to these two algorithms, we propose the PIONet model, which is 20× lighter compared to baseline models and retains equivalent performance. PIONet, along with Dynamic-Clustered-FedDC and Apriori-Clustered-FedDC, outperformed the state-of-the-art federated learning algorithms and the baseline.