Umeå University's logo

Federated Learning (FL) is a distributed machine learning paradigm that enables the training of models across numerous clients or organizations without requiring the transfer of local data. This method addresses concerns about data privacy and ownership by keeping raw data on the client itself and only sharing model updates with a central server. Despite its benefits, federated learning faces unique challenges, such as data heterogeneity, computation and communication overheads, and the need for personalized models. Thereby results in reduced model performance, lower efficiency, and longer training times.

This thesis investigates these issues from theoretical, empirical, and practical application perspectives with four-fold contributions, such as federated feature selection, adaptive client selection, model personalization, and socio-cognitive applications. Firstly, we addressed the data heterogeneity problems for federated feature selection in horizontal FL by developing algorithms based on mutual information and multi-objective optimization. Secondly, we tackled system heterogeneity issues that involved variations in computation, storage, and communication capabilities among clients. We proposed a solution that ranks clients with multi-objective optimization for efficient, fair, and adaptive participation in model training. Thirdly, we addressed the issue of client drift caused by data heterogeneity in hierarchical federated learning with a personalized federated learning approach. Lastly, we focused on two key applications that benefit from the FL framework but suffer from data heterogeneity issues. The first application attempts to predict the level of autobiographic memory recall of events associated with the lifelog image by developing clustered personalized FL algorithms, which help in selecting effective lifelog image cues for cognitive interventions for the clients. The second application is the development of a personal image privacy advisor for each client. Along with data heterogeneity, the privacy advisor faces data scarcity issues. We developed a daisy chain-enabled clustered personalized FL algorithm, which predicts whether an image should be shared, kept private, or recommended for sharing by a third party.

Our findings reveal that the proposed methods significantly outperformed the current state-of-the-art FL  algorithms. Our methods deliver superior performance, earlier convergence, and training efficiency.

Horizontal Federated Learning exhibits substantial similarities in feature space across distinct clients. However, not all features contribute significantly to the training of the global model. Moreover, the curse of dimensionality delays the training. Therefore, reducing irrelevant and redundant features from the feature space makes training faster and inexpensive. This work aims to identify the common feature subset from the clients in federated settings. We introduce a hybrid approach called Fed-MOFS 1 , utilizing Mutual Information and Clustering for local feature selection at each client. Unlike the Fed-FiS, which uses a scoring function for global feature ranking, Fed-MOFS employs multi-objective optimization to prioritize features based on their higher relevance and lower redundancy. This paper compares the performance of Fed-MOFS 2 with conventional and federated feature selection methods. Moreover, we tested the scalability, stability, and efficacy of both Fed-FiS and Fed-MOFS across diverse datasets. We also assessed how feature selection influenced model convergence and explored its impact in scenarios with data heterogeneity. Our results show that Fed-MOFS enhances global model performance with a 50% reduction in feature space and is at least twice as fast as the FSHFL method. The computational complexity for both approaches is O( d 2 ), which is lower than the state-of-the-art.

The key challenge of personalized federated learning (PerFL) is to capture the statistical heterogeneity properties of data with inexpensive communications and gain customized performance for participating devices. To address these, we introduced personalized federated learning in multi-tier architecture (PerMFL) to obtain optimized and personalized local models when there are known team structures across devices. We provide theoretical guarantees of PerMFL, which offers linear convergence rates for smooth strongly convex problems and sub-linear convergence rates for smooth non-convex problems. We conduct numerical experiments demonstrating the robust empirical performance of PerMFL, outperforming the state-of-the-art in multiple personalized federated learning tasks.

In the era of big data and federated learning, traditional feature selection methods show unacceptable performance for handling heterogeneity when deployed in federated environments. We propose Fed-FiS, an information-theoretic federated feature selection approach to overcome the problem occur due to heterogeneity. Fed-FiS estimates feature-feature mutual information (FFMI) and feature-class mutual information (FCMI) to generate a local feature subset in each user device. Based on federated values across features and classes obtained from each device, the central server ranks each feature and generates a global dominant feature subset. We show that our approach can find stable features subset collaboratively from all local devices. Extensive experiments based on multiple benchmark iid (independent and identically distributed) and non-iid datasets demonstrate that Fed-FiS significantly improves overall performance in comparison to the state-of-the-art methods. This is the first work on feature selection in a federated learning system to the best of our knowledge.

In the modern electronic power system, energy management and load forecasting are important tasks. Energy management systems are designed concerning monitoring and optimizing the energy requirement in smart systems. This research work is divided into two parts. The first part will contain load forecasting and energy management in a smart grid. Load forecasting in the smart grid can be divided into three parts long-term, mid-term, and short-term load forecasting. The second part will describe energy usage optimization for the electric vehicle. Here we will show grids to vehicle energy demand management and optimization. This chapter will first introduce different deep learning techniques and then discuss their applications related to smart-grid and smart vehicle.

Data plays a vital role in deep learning model training. In large-scale medical image analysis, data privacy and ownership make data gathering challenging in a centralized location. Hence, federated learning has been shown as successful in alleviating both problems for the last few years. In this work, we have proposed multi-diseases classification from chest-X-ray using Federated Deep Learning (FDL). The FDL approach detects pneumonia from chest-X-ray and also identifies viral and bacterial pneumonia. Without submitting the chest-X-ray images to a central server, clients train the local models with limited private data at the edge server and send them to the central server for global aggregation. We have used four pre-trained models such as ResNet18, ResNet50, DenseNet121, and MobileNetV2, and applied transfer learning on them at each edge server. The learned models in the federated setting have compared with centrally trained deep learning models. It has been observed that the models trained using the ResNet18 in a federated environment produce accuracy up to 98.3%98.3% for pneumonia detection and up to 87.3% accuracy for viral and bacterial pneumonia detection. We have compared the performance of adaptive learning rate based optimizers such as Adam and Adamax with Momentum based Stochastic Gradient Descent (SGD) and found out that Momentum SGD yields better results than others. Lastly, for visualization, we have used Class Activation Mapping (CAM) approaches such as Grad-CAM, Grad-CAM++, and Score-CAM to identify pneumonia affected regions in a chest-X-ray.

Cyber-physical disaster systems (CPDS) are a new cyber-physical application that collects physical realm measurements from IoT devices and sends them to the edge for damage severity analysis of impacted sites in the aftermath of a large-scale disaster. However, the lack of effective machine learning paradigms and the data and device heterogeneity of edge devices pose significant challenges in disaster damage assessment (DDA). To address these issues, we propose a generative adversarial network (GAN) and a lightweight, deep transfer learning-enabled, fine-tuned machine learning pipeline to reduce overall sensing error and improve the model's performance. In this paper, we applied several combinations of GANs (i.e., DCGAN, DiscoGAN, ProGAN, and Cycle-GAN) to generate fake images of the disaster. After that, three pre-trained models: VGG19, ResNet18, and DenseNet121, with deep transfer learning, are applied to classify the images of the disaster. We observed that the ResNet18 is the most pertinent model to achieve a test accuracy of 88.81%. With the experiments on real-world DDA applications, we have visualized the damage severity of disaster-impacted sites using different types of Class Activation Mapping (CAM) techniques, namely Grad-CAM++, Guided Grad-Cam, & Score-CAM. Finally, using k-means clustering, we have obtained the scatter plots to measure the damage severity into no damage, mild damage, and severe damage categories in the generated heat maps.

Lifelog images are very useful as memory cues for recalling past events. Estimating the level of event memory recall induced by a given lifelog image (event memorability) is useful for selecting images for cognitive interventions. Previous works for predicting event memorability follow a centralized model training paradigm that requires several users to share their lifelog images. This risks violating the privacy of individual lifeloggers. Alternatively, a personal model trained with a lifelogger’s own data guarantees privacy. However, it imposes significant effort on the lifelogger to provide a large enough sample of self-rated images to develop a well-performing model for event memorability. Therefore, we propose a clustered personalized federated learning setup, FedMEM, that avoids sharing raw images but still enables collaborative learning via model sharing. For an enhanced learning performance in the presence of data heterogeneity, FedMEM evaluates similarity among users to group them into clusters. We demonstrate that our approach furnishes high-performing personalized models compared to the state-of-the-art.

Sharing privacy-sensitive information poses a significant risk when private information is carelessly exposed to individuals or groups through various media. With minimal annotation and user preference, users can decide whether to share or keep an image based on its privacy score. This approach helps reduce the likelihood of privacy breaches and makes it easier to share images with friends or on social media platforms. Federated Learning (FL) allows collaborative learning of models without sharing personal data, which is crucial for sharing privacy-sensitive information. However, there are several challenges associated with learning from private data. This paper proposes two FL algorithms, namely Dynamic-Clustered-FedDC and Apriori-Clustered-FedDC, to address key problems, such as data scarcity, data heterogeneity, and model complexity, in order to ensure personalized image privacy. Both algorithms train personalized models for each annotator using clustered federated learning to address data heterogeneity. Additionally, both algorithms utilize daisy-chaining-based knowledge sharing between annotators to mitigate data scarcity issues during training. In addition to these two algorithms, we propose the PIONet model, which is 20× lighter compared to baseline models and retains equivalent performance. PIONet, along with Dynamic-Clustered-FedDC and Apriori-Clustered-FedDC, outperformed the state-of-the-art federated learning algorithms and the baseline.

Federated Learning (FL) has evolved as a promising distributed learning paradigm in which data samples are disseminated over massively connected devices in an IID (Identical and Independent Distribution) or non-IID manner. FL follows a collaborative training approach where each device uses local training data to train local models, and the server generates a global model by combining the local model's parameters. However, FL is vulnerable to system heterogeneity when local devices have varying computational, storage, and communication capabilities over time. The presence of stragglers or low-performing devices in the learning process severely impacts the scalability of FL algorithms and significantly delays convergence. To mitigate this problem, we propose Fed-MOODS, a Multi-Objective Optimization-based Device Selection approach to reduce the effect of stragglers in the FL process. The primary criteria for optimization are to maximize: (i) the availability of the processing capacity of each device, (ii) the availability of the memory in devices, and (iii) the bandwidth capacity of the participating devices. The multi-objective optimization prioritizes devices from fast to slow. The approach involves faster devices in early global rounds and gradually incorporating slower devices from the Pareto fronts to improve the model's accuracy. The overall training time of Fed-MOODS is 1.8× and 1.48× faster than the baseline model (FedAvg) with random device selection for MNIST and FMNIST non-IID data, respectively. Fed-MOODS is extensively evaluated under multiple experimental settings, and the results show that Fed-MOODS has significantly improved model's convergence and performance. Fed-MOODS maintains fairness in the prioritized participation of devices and the model for both IID and non-IID settings.

The challenge of personalized federated learning (pFL) is to capture the heterogeneity properties of data with in-expensive communications and achieving customized performance for devices. To address that challenge, we introduced personalized multi-tier federated learning using Moreau envelopes (pFedMT) when there are known cluster structures within devices. Moreau envelopes are used as the devices’ and teams’ regularized loss functions. Empirically, we verify that the personalized model performs better than vanilla FedAvg, per-FedAvg, and pFedMe. pFedMT achieves 98.30% and 99.71% accuracy on MNIST dataset under convex and non-convex settings, respectively.

Federated learning (FL) has gained a lot of attention in recent years for building privacy-preserving collaborative learning systems. However, FL algorithms for constrained machine learning problems are still limited, particularly when the projection step is costly. To this end, we propose a Federated Frank-Wolfe Algorithm (FedFW). FedFW features data privacy, low per-iteration cost, and communication of sparse signals. In the deterministic setting, FedFW achieves an ε-suboptimal solution within O(ε^-2) iterations for smooth and convex objectives, and O(ε^-3) iterations for smooth but non-convex objectives. Furthermore, we present a stochastic variant of FedFW and show that it finds a solution within O(ε^-3) iterations in the convex setting. We demonstrate the empirical performance of FedFW on several machine learning tasks.