The Jishi platform is a professional visual algorithm development and distribution platform. Join the Jishi professional CV group to interact with 6000+ visual developers from Tencent, Huawei, Baidu, Peking University, Tsinghua University, and the Chinese Academy of Sciences! You will also have the opportunity to interact with Professor Kai-Fu Lee and other experts!
We also provide monthly live sharing sessions with industry experts, real project demand matching, and a collection of valuable information for industry technical exchanges. Click “Read the Original” at the end of the article to apply to join the group now~
Source: David 9’s Blog
The sniper has undergone extensive training on small targets before adjusting the magnification, which seems to indicate that RPN performs well — David 9
Previously, when discussing SSD, we talked about how SSD’s object detection improves the detection rate of multi-scale (both large and small) objects. Let’s review: first, a larger convolution window can see larger objects after convolution, while a smaller one can only see smaller images. Imagine using an 11-sized minimum convolution window; the final convolved image’s granularity is exactly the same as the input image’s granularity. However, if you use a convolution window sized according to the image’s width and height, it can only encode a coarse output feature.
For YOLOv1, using the same size convolution window at each layer makes it powerless to recognize either extremely large or extremely small objects (the output feature map of the last layer is fixed at 7×7):
YOLO Architecture DiagramWhereas SSD goes a step further, the detection in the last layer is generated by multiple scale (Multi-Scale) feature maps:
SSD Architecture DiagramThus, SSD improves detection of multi-scale objects within the limits of computational complexity. However, SSD also has obvious flaws; its so-called “multi-scale” in the last few layers is limited (as shown in the above figure, the smaller the feature map size, the larger the objects that can be recognized). For extremely small object recognition, SSD becomes ineffective.
From:https://techcrunch.com/2017/06/16/object-detection-api/Assuming that a “kite” occupies only a few dozen pixels of the original image, the high-level feature map of SSD can no longer capture such small objects.
To solve the above problems, Facebook’s team developed FPN (Feature Pyramid Network). This network does not simply downsample to extract semantic features for object recognition but performs upsampling from each layer at the top (from top to bottom) to obtain more accurate pixel location information (similar to the skip connections in residual networks):
From:https://medium.com/@jonathanhui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106cWe know that convolution operations, while efficiently extracting semantics, also have pixel misalignment issues (by the time you reach the top feature map, you can no longer distinguish which pixels correspond to the original image). This method of upsampling to restore feature maps alleviates the pixel accuracy problem well (so that the pixels in the high-level feature map can be traced back):
From:https://medium.com/@jonathanhui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106cTherefore, to detect small targets, FPN also makes sacrifices, and the resulting issue is obvious: increased computational load (due to additional upsampling and skip layer calculations).
To alleviate such computational pressure and to identify multi-scale objects more accurately, SNIPER takes a different approach by maintaining a focus region for each large category of scales. Just like a sniper adjusting the magnification, the target of focus in the frame is at the optimal scale:
From: SNIPER: Efficient Multi-Scale TrainingThe dashed box in the above image indicates the focus region at a certain scale, while the solid box is the ground truth target box. It is clear that SNIPER attempts to enclose the ground truth within the appropriately scaled focus region. Additionally, SNIPER includes an exclusion area within the focus region, as many targets in the background do not need to be identified:
From: SNIPER: Efficient Multi-Scale Training.The green box in the image indicates ground truth, while the red box indicates the exclusion area, where the objects do not need to be recognized. Unlike FPN, SNIPER does not need to process the pixels of each layer’s feature map for upsampling, significantly reducing computational load. It is said to only process 30% more pixels than a typical one-shot model like YOLO (focusing on the overlapping areas within the focus region). Moreover, for simpler images, SNIPER theoretically allows degradation to a one-shot model similar to YOLO (by maintaining the focus region). In practice, SNIPER has been used in both RPN networks and Faster-RCNN.
References:
-
Machine Vision Object Detection Supplement: Real-time Detection with SSD, Multibox Single Shot Detector
-
Feature Pyramid Networks for Object Detection
-
https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c
-
SNIPER: Efficient Multi-Scale Training
-
https://github.com/mahyarnajibi/SNIPER
This article is licensed under the Attribution-NonCommercial-NoDerivatives 3.0 China Mainland License. Copyright belongs to “David 9’s Blog”. For reprints, please contact via email: [email protected]
Original:
http://nooverfit.com/wp/%E8%81%8A%E8%81%8A%E7%9B%AE%E6%A0%87%E6%A3%80%E6%B5%8B%E4%B8%AD%E7%9A%84%E5%A4%9A%E5%B0%BA%E5%BA%A6%E6%A3%80%E6%B5%8B%EF%BC%88multi-scale%EF%BC%89%EF%BC%8C%E4%BB%8Eyolo%EF%BC%8Cssd%E5%88%B0fpn/
*Recommended Reading*
-
Real-Time Machine Learning with TensorFlow in Data Collectors
-
Defect Detection Insights with 8500+ Views! Global Champion Team’s Comprehensive Analysis of Deep Learning Applications in Industrial Inspection (PPT + Video)
-
New Ideas on Gap Detection Algorithms
Monthly Live Sharing with Industry Experts, Real Project Demand Matching, Valuable Information Collection, Industry Technical Exchange. Click the bottom left corner “Read the Original” to apply to join the group now~
