Introduction
With the emergence of large model technologies represented by ChatGPT and Sora, the use of Transformer-based models for accurate pneumonia diagnosis has become a new research hotspot. The complex shapes, uneven gray distribution, and indistinct boundaries with surrounding tissues of lung X-ray images hinder the model’s ability to adequately extract features from lesions, thus restricting further improvements in pneumonia diagnostic accuracy. Professor Zhou Tao‘s team from the School of Computer Science and Engineering at Northern Minzu University proposed a dual residual Res-Transformer model for pneumonia auxiliary diagnosis, which was published as a cover article titled “DRT Net: A Dual Residual Res-Transformer Model for Feature Enhancement in Pneumonia Recognition” in Optics and Precision Engineering (an EI and Scopus indexed, core Chinese journal, classified as ‘T1’ in both the high-quality scientific journal grading directory in the field of instrumentation and the high-quality scientific journal grading directory in the field of optics and optical engineering).The first author of the paper is Professor Zhou Tao, and the corresponding author is Peng Caiyue, a graduate student from the School of Computer Science and Engineering at Northern Minzu University.
Research Background
Pneumonia, a common chest disease, is a respiratory infection caused by microorganisms and other environmental factors. It leads to fluid accumulation in the lungs and breathing difficulties, posing serious harm to the respiratory system. Traditional pneumonia diagnostic methods mainly rely on physicians’ years of clinical experience, which may introduce subjective errors and are difficult to implement on a large scale. Analyzing patients’ lung imaging features using deep learning technology can reduce the burden of complex medical data on healthcare professionals, improve diagnostic efficiency, and assist doctors in quickly forming targeted treatment plans. Furthermore, the diverse and complex characteristics of lung X-ray image lesions, along with limited contrast and unclear boundaries with surrounding tissues, hinder the model’s ability to focus on the lesion areas in the images, making it challenging to extract effective features for classification.
Figure 1: Various Lung X-ray Images
DRT Net
To effectively utilize global and local lesion area features in X-ray images and enhance the model’s recognition ability for different types of pneumonia, this paper proposes the DRT Net model, which is a dual residual Res-Transformer model aimed at feature enhancement. The overall structure of the model is shown in Figure 1, which includes a group attention dual residual module (A), a global-local feature extraction module (B), and a global-local feature extraction module (C). DRT Net leverages the global feature extraction capability of the Res-Transformer structure for pneumonia images along with local feature extraction capabilities, while incorporating three different feature enhancement strategies to improve the model’s perception of lesion areas.
Figure 2: Overall Framework of DRT NetThe basic residual network has limited performance and insufficient feature perception for images. Therefore, this paper designs a group attention dual residual module (GADRM). As shown in Figure 2, first, a dual residual structure is constructed, which combines two different types of residual connections to repeatedly mine and utilize features, enhancing the module’s ability to extract features from lesion areas. Secondly, a channel mixing operation is used to fuse the channel information of the feature maps after group convolution, dividing the fused feature map groups into four subgroups and performing different feature transformations; then, depthwise separable convolutions are used to extract and fuse features from each group, improving the recognizability of features within the group; finally, the group attention feature extraction block is constructed by combining squeeze-and-excitation operations with spatial attention operations, enhancing the module’s feature expression capability. 
Figure 3: Group Attention Dual Residual ModuleWhile convolution operations extract features, they also lose low-level texture details, resulting in high-level features and low-level features being distributed at opposite ends of the network. High-level features contain stronger semantic information but have lower resolution and poorer detail perception; shallow features have high resolution, containing more positional details, edges, textures, etc. Additionally, the information that different layers of the feature extraction network focus on varies, and using features from different layers to fuse contextual information can enhance the network’s classification performance. This paper designs a cross-layer dual attention feature fusion module (CDAFFM) to compensate for the lack of deep semantic information with shallow semantic information. Low semantic information such as texture and shape from shallow networks is enhanced using spatial attention, while high semantic information from deep networks is enhanced through channel attention. The filtered channel and spatial information are summed, allowing efficient fusion of shallow and deep contextual information in the images, retaining more useful information and improving the model’s classification performance.
Figure 4: Cross-Layer Dual Attention Feature Fusion Module
Experimental Results
To evaluate the effectiveness of the modules, various network models were tested to assess the performance of each module, with evaluation metrics including accuracy, macro-average precision, macro-average recall, macro-average F1 score, and AUC value. The accuracy of our model is 98.41%, precision is 94.42%, recall is 94.20%, F1 score is 94.26%, and AUC value is 99.65%. To visually compare the results of each experiment, an ablation study radar chart was drawn, as shown in Figure 5, where the proposed DRT Net line is located at the outermost side, indicating optimal model performance.
Figure 5: Ablation Study Results Radar Chart
Future Prospects
In recent years, artificial intelligence has become an important means of disease diagnosis using medical images. The use of computer-aided diagnosis methods to solve medical problems is an inevitable trend for future development. Medical image classification technology can enhance the accuracy and efficiency of disease diagnosis for patients, promoting the development of intelligent disease auxiliary diagnosis in the medical industry, and revitalizing the healthcare industry through the application of artificial intelligence technology.
Team Introduction
The team relies on the Key Laboratory of Image and Graphics Intelligent Processing of the National Ethnic Affairs Commission and the Innovative Team of Medical Information Intelligent Perception and Advanced Computing. They have long been engaged in computer-aided diagnosis research for lung diseases, oral diseases, and mandible fractures based on medical imaging. This includes medical image classification and recognition, medical image segmentation, cross-modal medical image fusion, and lesion target detection, dedicated to in-depth research and application development of artificial intelligence technology in the field of medical imaging.Under the leadership of Professor Zhou Tao, the team has led over 20 various research projects and published more than 210 academic papers in high-level domestic and international journals such as Information Fusion, Applied Soft Computing, Journal of Electronics and Information, Journal of Electronics, and Optics and Precision Engineering, with over 100 papers indexed by SCI and EI, including 3 ESI highly cited papers, with a single paper citation exceeding 500 and over 11,000 downloads.Three monographs have been published by Science Press, and 13 Chinese invention patents and 4 Australian innovation patents have been applied for.
Team Achievements
[1] Zhou T, Niu Y, Lu H, et al. Vision transformer: To discover the “four secrets” of image patches[J]. Information Fusion, 2024, 105: 102248.
[2] Zhou T, Li Q, Lu H, et al. GAN review: Models and medical image fusion applications[J]. Information Fusion, 2023, 91: 134-148.
[3] Zhou Tao, Liu Yuncan, Lu Huiling, Ye Xinyu, Chang Xiaoyu. ResNet and its applications in medical image processing: Research progress and challenges[J]. Journal of Electronics and Information, 2022, 44(1): 149-167.
[4] Zhou T, Liu F, Ye X, et al. CCGL-YOLOV5: A cross-modal cross-scale global-local attention YOLOV5 lung tumor detection model[J]. Computers in Biology and Medicine, 2023, 165: 107387.
[5] Zhou Tao, Dang Pei, Lu Huiling, et al. Cross-modal cross-scale cross-dimensional PET/CT image Transformer segmentation model[J]. Journal of Electronics and Information, 2023, 45(10): 3529-3537.
Paper Information
Zhou Tao, Peng Caiyue, Du Yuhu, et al. DRT Net: A Dual Residual Res-Transformer Model for Feature Enhancement in Pneumonia Recognition[J]. Optics and Precision Engineering, 2024, 32(05): 714-726. DOI: 10.37188/OPE.20243205.0714.https://ope.lightpublishing.cn/zh/article/doi/10.37188/OPE.20243205.0714/
Supervised by: Cao Jin, Zhao Yang
Edited by: Zhao Wei
👇 Follow Me 👇 Hit “Like” and “See” so you won’t miss any article updates