Original Information
Paper Title:Model Similarity Based Clustering Federated Learning in Edge Computing
Accepted Conference:EAI CollaborateCom 2024 (CCF C)
Author List
1) Liu Xiaoyan, China University (Beijing), School of Artificial Intelligence, PhD student of 2023
2) Huang Jiwei, China University (Beijing), School of Artificial Intelligence, Professor
3) Chen Ying, Beijing Information Science and Technology University, School of Computer Science, Professor
Abstract
With the development of edge computing, Federated Learning (FL) has become an important distributed machine learning framework to ensure data privacy and reduce communication overhead. However, traditional Clustered Federated Learning (CFL) has limitations when dealing with Non-Independent and Identically Distributed (non-IID) data, especially since the aggregation weights often depend on the size of the client datasets, neglecting their actual contribution to the global model. To address this, this paper proposes a Model Similarity-based Clustering Federated Learning framework (MS-CFL). This method clusters clients based on model parameter similarity and introduces Deep Reinforcement Learning (DRL) to dynamically optimize aggregation weights, balancing data scale and model accuracy. Experimental results show that MS-CFL outperforms existing methods in terms of accuracy and communication efficiency.
Background and Motivation
In the Edge Computing (EC) environment, a large number of terminal devices can directly participate in the training of intelligent tasks, but their computing power and storage capacity are limited, and the data distribution often exhibits a high degree of non-IID characteristics. Non-IID data can lead to significant differences in local optimization directions among different clients, making global model aggregation difficult and severely degrading performance. Traditional CFL methods often use the size of client datasets as aggregation weights, which often overlooks the differences in contributions of client models to global performance. Additionally, in reality, clients may be located in overlapping coverage areas of base stations, and a lack of reasonable clustering and base station selection strategies can further increase communication overhead and reduce model convergence efficiency. Therefore, the motivation of this work is to:
(1) Achieve more reasonable client clustering through model parameter similarity;
(2) Optimize weight distribution through DRL to balance data scale and contribution;
(3) Design a hierarchical federated learning architecture to improve aggregation efficiency and reduce communication overhead.
Main Content
Figure 1: Clustering Federated Learning Architecture.
The proposed MS-CFL framework enhances the performance and communication efficiency of federated learning in non-IID data scenarios by introducing model similarity clustering and reinforcement learning-driven weight optimization in the edge computing environment. We consider the following three-layer hierarchical architecture, as shown in Figure 1. Specifically, the client layer (Client) is where terminal devices perform local model training; the base station layer (Base Station) aims to aggregate client models within the coverage of each base station; the edge server layer (Edge Server) acts as the global coordinator, responsible for final aggregation and parameter distribution, while dynamically adjusting weight distribution using DRL. The specific algorithm is as follows:
Secondly, we designed a clustering method based on model similarity. After completing local model training, clients upload their model parameters. To measure the distribution differences of model parameters among different clients, we calculate the similarity between client models using cosine similarity:
For clients located in overlapping coverage areas of base stations, we select the optimal base station based on similarity to achieve reasonable clustering allocation.
The pseudocode is as follows:
Finally, we optimized the aggregation weights. In the global aggregation process, the aggregation weights are determined by both data scale and testing accuracy:
We use the DDQN algorithm for dynamic adjustment of variables, considering the testing accuracy and loss of each model group.
Experimental Results and Analysis
We designed different non-IID partitions on multiple public datasets and conducted extensive simulations to verify the effectiveness of the proposed framework and the superiority of the algorithm. The experimental results are shown in Tables 1 and 2: Table 1 presents the testing accuracy of each method on the global model under different scenarios, while Table 2 shows the comparison of average testing accuracy among clients. Furthermore, Figures 2 and 3 compare the number of communication rounds required for each method to reach a specific testing accuracy for client models in non-IID scenarios, highlighting the advantages of the proposed method in terms of communication efficiency.
Figure 2: Average Testing Accuracy of Clients (NIID-3)
Figure 3: Average Testing Accuracy of Clients (NIID-4)
Conclusion
The proposed MS-CFL framework significantly improves the accuracy and efficiency of federated learning in edge computing by clustering based on model similarity and optimizing weights with DRL, while addressing the challenges of non-IID data. Experimental results indicate that MS-CFL outperforms comparative methods across different datasets and non-IID scenarios. Future work will further explore its extensions in security and robustness, and validate it on larger-scale and more diverse real-world datasets.
About the Author
Huang Jiwei, Professor
PhD supervisor, Vice Dean of the School of Artificial Intelligence at China University of Petroleum (Beijing), Director of the Beijing Key Laboratory of Petroleum Data Mining, Vice President of the Changping District Association of Young Talents. Selected as an outstanding talent in Beijing, a Beijing Science and Technology Star, a young talent in national governance in Beijing, a young talent in the Changju Project, and an excellent young scholar at China University of Petroleum (Beijing). He graduated with a bachelor’s and PhD from the Department of Computer Science and Technology at Tsinghua University and jointly trained PhD students at the Georgia Institute of Technology in the USA. His research interests include: Internet of Things, service computing, edge intelligence, etc. He has presided over 20 research projects including the National Natural Science Foundation, National Key R&D Program, and Beijing Natural Science Foundation; published over 70 academic papers as the first/corresponding author in well-known domestic and international journals and conferences, including 1 paper awarded the Excellent Paper Award by the China Association for Science and Technology, 3 papers selected as ESI Hot Papers, and 7 papers selected as ESI Highly Cited Papers; published 1 academic monograph; obtained 6 national invention patents and 4 software copyrights; received 1 first prize in scientific and technological achievements from the China Communications Society, 1 first prize in innovation achievements from the China Industry-University-Research Cooperation, 1 second prize in scientific and technological achievements from the Guangdong Computer Society, and 1 CCF Service Computing Young Talent Award. He serves as a member of the Service Computing Committee of the China Computer Federation (CCF), a senior member of CCF and IEEE, and an editorial board member of journals such as the Journal of Electronics and Scientific Programming.
Contact: [email protected]
Scan QR Code ｜ Follow Us
The images in this article are sourced from the original text. If there is any infringement, please contact us for removal.