Table of Contents | 2023 Issue 3 Special Topic: Deep Integration of 6G Sensing and Computing
01Mobile Communications 2023 Issue 2Special Topic on “Key Technologies and Applications of 6G”
Smart 6G: Edge Deployment and Lightweight Networks*
Zhou Ziyao, Liu Qingling, Tao Jianying, Lin Yun
(School of Information and Communication Engineering, Harbin Engineering University, Harbin, Heilongjiang 150001, China)
*Funding Projects: National Natural Science Foundation General Project (62201172); Basic Scientific Research Business Fee of Central Universities (3072022CF0804)
【Abstract】6G mobile communication technology will advance human society into the intelligent era, achieving the integration of the digital world and the real world. In this context, deploying artificial intelligence models on edge computing platforms has become an inevitable trend, with reducing model size and improving model performance becoming key research directions. This article summarizes the research on edge computing and model lightweighting from a broad perspective, providing comprehensive background knowledge for related scholars, and discusses the importance of the combination of edge computing and lightweight models by analyzing reliable data to forecast the development prospects of this field.
【Keywords】6G; Edge Computing; Mobile Edge Computing; Knowledge Distillation; Network Pruning
doi:10.3969/j.issn.1006-1010.20230105-0002
Classification Number: TN929.5 Document Mark Code: A
Article Number: 1006-1010(2023)02-0002-06
Citation Format: Zhou Ziyao, Liu Qingling, Tao Jianying, et al. Smart 6G: Edge Deployment and Lightweight Networks[J]. Mobile Communications, 2023, 47(2): 02-07.
ZHOU Ziyao, LIU Qingling, TAO Jianying, et al. Smart 6G: Edge Deployment and Lightweight Networks[J]. Mobile Communications, 2023, 47(2): 02-07.
0 Introduction
The global mobile communication technology has entered the 5G era, and commercial 5G technology can support low-latency and high-reliability IoT services, making the vision of cloud-based intelligence reaching user terminals a reality. However, with the rapid development of artificial intelligence, communication technology, hardware devices, and other fields, users have also proposed higher-level service demands[1]. Based on this, 6G will evolve from typical 5G scenarios, introducing super unlimited bandwidth, ultra-large scale connectivity, and extremely reliable communication, while expanding new service scenarios such as universal intelligence and communication-sensing integration[2]. These service scenarios will generate massive amounts of data, and the traditional cloud computing framework, which centrally processes edge data, is not only inefficient and energy-consuming but also reduces service quality due to latency and other issues, leading to the emergence of edge computing (EC).
EC is a technology that can analyze and process data at the network edge. By deploying edge computing platforms, local terminals can directly connect to nearby servers for real-time processing, minimizing the amount of data uploaded to the cloud, freeing up bandwidth, and reducing service costs. At the same time, with the reduction of physical distance, data transmission latency will also be greatly reduced, providing users with better service quality.
Enhanced edge computing has another development direction—artificial intelligence (AI). In recent years, AI technology has made continuous breakthroughs, but with the improvement in performance, the size and energy consumption of models have also been increasing. The current edge terminals (mostly sensors) usually operate at mW levels of power consumption, and for most of the time, they will enter a sleep state at μW levels of power consumption. Ensuring model performance under limited resources is a challenging task, and model lightweighting techniques, including knowledge distillation and structure pruning, have sparked a new wave of research. In this context, the organic combination of edge computing and model lightweighting, along with the popularization of intelligent services on edge devices, will become a mainstream trend in commercial development. This paper focuses on discussing and analyzing the basic theories, key technologies, and development status of edge computing and model lightweighting.
1 Edge Node Network Deployment
Given the complexity, advancement, and leading role of 6G systems in the industry, major developed countries have increased their investment and conducted in-depth research to seek competitive advantages. In the 5G era, deep learning data processing typically can only be performed in cloud computing centers, and this system has four shortcomings when processing edge data: (1) High latency, making it difficult to meet the real-time requirements of services; (2) Excessive data flow, which can lead to channel congestion; (3) High energy consumption, which increases operational costs; (4) Low security, with significant risks of privacy data leakage. Based on the 5G IoT, a “cloud-network integration” (architecture as shown in Figure 1) is established, where the cloud processor controls network resource allocation and real-time scheduling; achieving network cloudification: opening the door to traditional closed networks, realizing local cloudification and intelligence, and achieving goals such as elastic resource allocation, rapid networking, and intelligent control[3]. This cloud-centric server model, with the establishment of edge nodes and technical upgrades and optimization management of edge servers, can effectively solve existing problems and represents a promising development direction. This section will introduce the relevant theories of edge computing and analyze the advantages of using edge computing in 6G networks through data.
1.1 Edge Computing (EC)
Edge computing is a new computing paradigm proposed by Google in 2006, which executes computing tasks at the edge of the network. Compared to cloud computing, edge computing is closer to the source of data, which can reduce latency during data transmission to a certain extent; at the same time, because data is processed nearby, there is no need to upload all data to cloud servers, which can better protect privacy data from leakage and minimize the misuse of information; additionally, it can significantly reduce communication energy consumption and relieve the pressure on network bandwidth.
Edge computing mainly introduces edge nodes between terminal devices and cloud servers, determining the task categories of users at the nodes, delivering tasks that require less timeliness, are data-intensive, and need in-depth analysis to the cloud center server for processing; while allowing small-scale, real-time intelligent analysis tasks to reside at the edge nodes for processing. Therefore, in intelligent services, tasks that are data-intensive and require large-scale centralized processing can continue to use the cloud centralized processing model, while edge computing is more suitable for small-scale data analysis tasks and local services[4]. The architecture of edge computing is shown in Figure 2:
In recent years, various sensing devices have been continuously upgraded, and the data collection capabilities have significantly improved, allowing for the detection of real-time information during most industrial production processes. For the massive data generated during the detection process, traditional processing models face issues such as high latency and high costs, which cannot meet industrial demands. Given the advantages of edge computing, it can be widely applied in industrial production links[5]. Typical application scenarios include: deep learning-based bearing fault diagnosis[6] and power equipment maintenance[7], where edge computing is used for fault diagnosis and defect detection; comprehensive management of drones for large-scale data collection and real-time monitoring[8], thus enhancing security management in industrial parks; and during product evaluations, the use of EC’s virtual reality and augmented reality (VR/AR) technologies has made data analysis more efficient for workers.
Although edge computing has been applied in numerous scenarios, there are still many issues that need further improvement, including low EC performance, insufficient security, and poor collaboration protocols among devices. Therefore, to achieve the vision of 6G networks and widely popularize intelligent services, these issues need to be continuously addressed and researched.
1.2 Mobile Edge Computing (MEC)
The concept of Mobile Edge Computing was officially proposed by the European Telecommunications Standards Institute (ETSI) in 2016, which adds certain computing capabilities to base stations, while edge computing also attaches computing capabilities to private clouds, Cloudlets, and other locations, which is the biggest difference between the two. The network deployment diagram of MEC and EC is shown in Figure 3:
MEC constructs a service environment at the Radio Access Network (RAN) layer, allowing some services to detach from the core network, thereby saving resources, reducing latency, and improving service quality. According to the white paper released by ETSI, MEC has the following characteristics[9]:
(1) Local Services: MEC can operate independently of the rest of the internet and access local network resources, which is very important for M2M (Machine-to-Machine) scenarios. Additionally, due to the isolation characteristics between MEC and other networks, data security can be effectively enhanced.
(2) Proximity: MEC is deployed at the nearest location to mobile terminals, providing significant advantages for certain computation-intensive devices (e.g., augmented reality, video analysis).
(3) Low Latency: MEC is deployed close to user devices, which can minimize the interaction process between local data and cloud data, greatly reducing user service latency, while the number of users within the coverage remains relatively stable, alleviating bandwidth pressure on servers.
(4) Location Awareness: Edge-distributed devices share information in real-time through low-level signaling, allowing MEC to receive data from local network devices, enabling device positioning.
(5) Network Context Information: Applications providing network information and real-time internet data services can utilize MEC with RAN real-time information to analyze network status, assisting operators in making reasonable judgments and providing users with higher quality services.
The technical characteristics of MEC, such as “proximity, low latency, and location awareness,” can meet many demands in 6G service scenarios (as shown in Table 1), bringing extensive application prospects.
As shown in Table 1, MEC can be applied to various service scenarios such as the Internet of Vehicles (IoV), Virtual Reality/Augmented Reality (VR/AR), Smart Healthcare, Public Safety, and Smart Cities. In the IoV, combining MEC can provide reasonable suggestions to drivers by analyzing environmental information and interacting with other vehicles in the network during vehicle travel, eliminating potential hazards and avoiding traffic congestion risks; smart healthcare requires real-time processing of large amounts of data, and applying MEC can provide stronger technical support for remote medical services; smart home systems can utilize numerous IoT devices for real-time data monitoring, combined with MEC to schedule the usage status of home devices, thereby improving the safety, convenience, and comfort of contemporary homes[10].
MEC further refines edge computing, integrating MEC with artificial intelligence, sensing, and detection technologies, continuously conducting in-depth research, adapting to new scenarios, enhancing user experience, and thus expanding the applicability of 6G services, providing a brighter future for intelligent home access.
2 Network Model Lightweighting Based on User Terminals
The previous section discussed the edge deployment of 6G intelligent networks, but edge nodes are merely an intermediary between user terminals and cloud servers. For some small data analysis tasks, terminal devices still need to process them independently. In traditional neural network models, previous scholars have focused more on improving model recognition accuracy and adaptability in complex environments, resulting in increased model parameters and complexity, leading to skyrocketing energy costs. In recent years, thanks to the rapid development of AI technology, various industries have gradually shifted their focus to the application of artificial intelligence in real life. Due to limited storage resources, low processor performance, and energy constraints on platforms such as mobile and embedded devices, most high-precision models cannot be deployed on user terminals and are difficult to run in real-time. This necessitates that traditional deep neural network models reduce parameter counts and complexity while ensuring performance, making model lightweighting a research hotspot[11].
2.1 Knowledge Distillation (KD)
Knowledge distillation is an emerging model compression method that allows a smaller, simpler student model to learn from a complex teacher model while maintaining performance. This method can be viewed as a form of transfer learning, with the key issue being how to transfer the knowledge of the teacher model to the student model while ensuring that performance does not degrade or even improves during the transfer process; this requires deeper research into the transfer process.
The framework of knowledge distillation mainly consists of three parts, whose structure is shown in Figure 4. In this framework, knowledge is distilled from the results output by the teacher model and then further transformed and delivered to the lightweight student model, ensuring minimal performance fluctuation during this process. The most critical part of this process is the acquisition of “knowledge.” On one hand, this “knowledge” can be interpreted as certain similarities contained in the output data of the teacher model, which can be used for training other models; some literature refers to this as “dark knowledge”[12]; on the other hand, it can be understood as all features, parameters, and other data generated by the teacher model that can be utilized by other models, with the “distillation” process being the discovery and amplification of these similarities for training other models[13].
In recent years, researchers have conducted numerous experiments, and from the data in Table 2, it can be seen that the size of the distilled model has been significantly reduced while improving model efficiency without sacrificing accuracy.
Moreover, many bottleneck issues in mainstream technologies can be solved through knowledge distillation, with the following examples:
(1) Generative Adversarial Networks (GANs) are often complex in computation and require high storage resources, making it difficult to deploy directly on mobile devices; combining knowledge distillation can simplify the generator[14], discriminator[15], or both simultaneously[16]. Currently, although GANs combined with knowledge distillation have achieved many excellent results in compression, there are still some issues (such as difficulty in training and lack of interpretability) that require deeper research by scholars.
(2) Reinforcement Learning (RL) has applications for deep reinforcement learning models in many fields[17], such as robot control[18], complete information games[19], and incomplete information games[20]. However, this model requires extensive interaction with the external environment to update network parameters, resulting in significant training costs. Combining knowledge distillation can enhance training effectiveness and achieve model lightweighting.
(3) Federated Learning (FL) can maximize data privacy protection during training, breaking the data boundaries of traditional institutions and having very broad application prospects in reality. Federated Learning also supports local data training and transmits encrypted training data to servers, but excessive data volume often occupies a large bandwidth, generating high communication costs. Deploying knowledge distillation in various stages of federated learning can reduce the bandwidth required for distributed federated learning.
From the above examples, it is evident that knowledge distillation can reduce model size while ensuring performance, achieving the miniaturization of complex models, and thus being deployable on resource-limited edge devices[21], making intelligent home access a reality.
2.2 Neural Network Pruning
In recent years, neural network pruning has become widely recognized as an effective model compression method. Although most large neural networks have outstanding learning capabilities, data analysis during experimental processes can reveal that certain structures within the models are not functioning as intended, leading to the idea of neural network pruning: removing unnecessary model structures without affecting network performance, thereby reducing model size[22]. The specific approach is as follows: analyzing and evaluating the nodes within the network, removing nodes with unsatisfactory evaluation results, thus achieving model lightweighting (pruning effects are referenced in Table 3 data), allowing medium and large deep neural networks to be deployed on edge nodes and devices.
Neural network pruning mainly divides into two methods: structured pruning and unstructured pruning, where unstructured pruning mainly achieves its purpose by reducing weight parameters, while structured pruning includes other techniques such as convolution kernel pruning and channel pruning.
Unstructured pruning often requires additional information to improve sparsity while maintaining model performance. The parameters to be removed can be determined by the following methods: Optimal Brain Damage (OBD) establishes a local model of the error function using second-order derivatives, selectively deleting network weights[23]; Optimal Brain Surgeon (OBS) uses all second-order derivative information of the error function to perform pruning[24]; pruning indicators based on minimum contribution variance compare the differences in output data before and after bias parameters to remove connection parameters with minimal contribution variance[25].
Unstructured pruning only removes some parameters, and its performance in reducing computation and parameter counts is relatively mediocre. Structured pruning optimizes convolution kernels and channels, enhancing the performance of neural networks on an overall structural level. By deleting some convolution kernels that contribute little to the network, the network inference time can be effectively accelerated, such as selecting redundant convolution kernels for pruning based on the contribution of each convolution kernel to the output results[26]; considering the reduction in computation, filters with lower output precision can be removed to minimize matrix multiplication operations, such as compression techniques based on filter pruning[27]; by pruning channels with layer-related thresholds, important channels can be optimally separated from negligible channels, such as Optimal Thresholding (OT)[28].
Various methods mentioned above can minimize model size while maintaining performance, reducing computational load, and providing possibilities for mobile portable devices to host more artificial intelligence applications.
3 The Combination of Edge Nodes and Model Lightweighting in the 6G Ecosystem
With the continuous development of AI technology, people’s requirements for intelligent living will gradually increase, and the design route of AI will progress towards intelligence, flexibility, security, and ease of deployment. In the process of utilization, intelligent devices need to face four roles: (1) Device manufacturers; (2) Cloud vendors; (3) Communication vendors; (4) Algorithm suppliers. Building a complete 6G intelligent ecosystem requires cooperation and progress among all four parties. However, historical experience shows that the most important role is that of device manufacturers, while the other three parties always face a pain point—numerous algorithms are difficult to implement on hardware devices.
Combining the previously discussed edge computing and model lightweighting: lightweight processing is performed by the algorithm suppliers, deploying high-performance applications on edge devices, and coordinating all parties through edge nodes established by communication vendors, connecting to the cloud for task allocation, maximizing network efficiency. Through this approach, each party can complement each other, coordinating to a certain extent to address the shortcoming effect, promoting the development of the 6G intelligent ecosystem and the comprehensive realization of intelligent services.
4 Conclusion
This paper focuses on the prospects of 6G “cloud-network integration,” analyzing the development direction of intelligent services in 6G, emphasizing the discussion of basic principles, key technologies, and application prospects of edge computing, mobile edge computing, knowledge distillation, and structure pruning. Through data analysis, it addresses the pain points of the 6G intelligent ecosystem and explores solutions. However, with the in-depth study of 6G networks and protocol refinement, many issues are bound to arise. How to respond to emerging problems, enhance the scope of 6G services, and service capabilities, and promote the intelligence of future society still requires collaborative efforts from all scholars in the academic community.
References:(Scroll to browse)
[1] Li Long. Resource Joint Optimization for Multiple Mobile Devices’ Remaining Energy in MEC-WPT Systems[D]. Jilin: Jilin University, 2021.
[2] Han Shujun,Dong Qing,Xu Xiaodong. Multi-band Collaborative Coverage Extension Technology for All Scenarios in 6G[J]. ZTE Communications Technology, 2022,28(4): 31-35.
[3] Zhao Jinsong,Zhang Chaoyang,Gu Weifeng,et al.. Smart Electric Plant Platform Architecture Based on Industrial Internet[J]. Thermal Power Generation, 2019,48(9): 101-107.
[4] Cao K, Liu Y, Meng G, et al. An overview on edge computing research[J]. IEEE access, 2020,8: 85714-85728.
[5] Wang Qichao,Jin Guangshu,Li Qing,et al.. Current Status and Outlook of Industrial Edge Computing Research[J]. Information and Control, 2021,50(3): 257-274.
[6] Peng Chao,Tang Xianghong,Lu Jianguang. Bearing Fault Diagnosis Based on Edge Computing[J]. Combined Machine Tool and Automation Processing Technology, 2020,12: 52-55.
[7] Li Xing,Hou Wei,Yang Yu,et al.. Application and Practice of Edge Computing in Power Infrastructure Business[J]. Electrical Engineering Journal, 2020,15(4): 153-160.
[8] Salhaoui M, Antonio Guerrero-González, Arioua M, et al. Smart industrial IoT monitoring and control system based on UAV and cloud computing applied to a concrete plant[J]. Sensors, 2019,19(15): 3316．
[9] Tao Yaodong,Xu Wei,Ji Shenglong. Overview and Outlook on Edge Computing Security[J]. Computer Integrated Manufacturing Systems, 2019,25(12): 3043-3051.
[10] Liang Guangjun,Wang Qun,Xin Jianfang,et al.. Overview of Resource Allocation in Mobile Edge Computing[J]. Journal of Information Security, 2021,6(3): 227-256.
[11] Sun J, Zhong Q, Kou L, et al. A lightweight multi-factor mobile user authentication scheme[J]. IEEE INFOCOM 2018: IEEE Conference on Computer Communications Workshops, 2018: 831-836.
[12] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[EB/OL]. (2015-03-09)[2023-01-04]. https://doi.org/10.48550/arXiv.15 03.02531.
[13] Zhao Renrong,Liu Yuang,Zhang Wei,et al.. Overview of Knowledge Distillation in Deep Learning[J]. Journal of Computer Science, 2022,45(8): 1638-1673.
[14] Aguinaldo A, Chiang P Y, Gain A, et al. Compressing GANs using knowledge distillation[EB/OL]. (2019-02-01)[2023-01-04]. https://arxiv.org/abs/1902.0 0159.
[15] Chen H, Wang Y, Xu C, er al. Data-free learning of student networks[EB/OL]. (2019-12-31)[2023-01-04]. https://arxiv.org/abs/1904.01186v1.
[16] Chen H, Wang Y, Shu H, et al. Distilling portable generative adversarial networks for image translation[EB/OL]. (2020-03-07)[2023-01-04]. https://arx iv.org/abs/2003.03519
[17] Sutto R S,Barto A G. Reinforcement Learning: An Introduction[J]. IEEE Transactions on Neural Networks, 1998,9(5): 1054-1054.
[18] Levine S, Finn C, Darrell T, et al. End-to-end training of deep visuomotor policies[J]. The Journal of Machine Learning Research, 2016,17(1): 1334-1373.
[19] Silver D, Schrittwieser J, Simonyan K, et al. Mastering the game of go without human knowledge[J]. nature, 2017,550(7676): 354-359.
[20] Zha D, Lai K H, Cao Y, et al. A toolkit for reinforcement learning in card games[EB/OL]. (2020-02-14)[2023-01-04]. https://arxiv.org/abs/1910.04376.
[21] Y Tu, Y Lin. Deep Neural Network Compression Technique Towards Efficient Digital Signal Modulation Recognition in Edge Device[J]. IEEE Access, 2019, 7: 58113-58119.
[22] Y Lin, Y Tu, Z Dou, An Improved Neural Network Pruning Technology for Automatic Modulation Classification in Edge Devices[J]. IEEE Transactions on Vehicular Technology, 2020,69(5): 5703-5706
[23] Lecun Y. Optimal brain damage[J]. Neural Information Proceeding Systems, 1990,2(279): 598-605.
[24] Poppi R J, Massart D L. The optimal brain surgeon for pruning neural network architecture applied to multivariate calibration[J]. Analytica Chimica Acta, 1998,375(1-2): 187-195.
[25] ThiMM G, FiEsLER E. Evaluating pruning methods[J]. International Symposium on Artificial Neural Networks, 1995,A2: 20-25.
[26] Luo J H, Wu J X, Lin W Y. ThiNet: a filter level pruning method for deep neural network compression[EB/OL]. (2017-07-20)[2023-01-04]. https://arxiv. org/abs/1707.06342.
[27] Li H, Kadav A, Durdanovic I, et al. Pruning filters for efficient convnets[EB/OL]. (2017-3-10)[2023-01-04]. https://arxiv.org/abs/1608.08710.
[28] Luchetta A. Automatic generation of the optimum threshold for parameter weighted pruning in multiple heterogeneous output neural networks[J]. Neurocomputing, 2008,71(16-18): 3553-3560. ★
★Scan the QR code to read and download this paper on CNKI
★Original article published in Mobile Communications Issue 1, 2023★
doi:10.3969/j.issn.1006-1010.20230105-0002
Classification Number: TN929.5 Document Mark Code: A
Article Number: 1006-1010(2023)02-0002-06
Citation Format: Zhou Ziyao, Liu Qingling, Tao Jianying, et al. Smart 6G: Edge Deployment and Lightweight Networks[J]. Mobile Communications, 2023, 47(2): 02-07.
ZHOU Ziyao, LIU Qingling, TAO Jianying, et al. Smart 6G: Edge Deployment and Lightweight Networks[J]. Mobile Communications, 2023, 47(2): 02-07.
Author Introduction
Zhou Ziyao（orcid.org/0000-0002-5640-4421）：Master’s student at the School of Information and Communication Engineering, Harbin Engineering University, research direction in software-defined radio technology, deep learning, FPGA.
Liu Qingling：Ph.D., currently an associate professor and master’s supervisor at the School of Information and Communication Engineering, Harbin Engineering University, research direction in wireless network technology, internet information security, artificial intelligence, information technology, MANETs, etc.
Tao Jianying：Master’s student at the School of Information and Communication Engineering, Harbin Engineering University, research direction in communication technology, FPGA.
Lin Yun（orcid.org/0000-0003-1379-9301）：Currently a professor and doctoral supervisor at Harbin Engineering University, research direction in intelligent radio technology, artificial intelligence and machine learning, big data analysis and mining, software and cognitive radio, information security and countermeasures, intelligent information processing.
The submission method for Mobile Communications is online submission
Please log in to the web submission system
Link: http://ydtx.cbpt.cnki.net
HighlightsCall for Papers | Special Issue 8, 2023: New Technologies for IoT Aimed at 6GCall for Papers | Special Issue 10, 2023: Mobile Self-organizing NetworksCall for Papers | Special Issue 12: “Internet of Vehicles”2023 Special Plan for Mobile CommunicationsIssue 6, 2022 | Collection of 6G Special Papers (13 papers)
#Scan to follow us# Mobile Communications interprets communication with papers
Mobile Communications magazine is supervised by China Electronics Technology Group Corporation and sponsored by the Seventh Research Institute of China Electronics Technology Group Corporation. It is a “double-effect journal” in the Chinese journal matrix, a high-quality electronic journal of the Ministry of Industry and Information Technology, a source journal of Chinese science and technology papers, and is included in the “High-quality Science and Technology Journal Classification Directory” of the China Association for Science and Technology. Domestic continuous publication number: CN44-1301/TN, International continuous publication number: ISSN1006-1010, Postal delivery code: 46-181.