Real-Time Inference Detection of Industrial Defects (C++)

Real-Time Inference Detection of Industrial Defects (C++)Source Code

https://www.gitpp.com/robolao/project0828009-limr_cpp

Introduction to the Open Source Project for Real-Time Inference Detection of Industrial Defects (C++)

Project Background and Core Value

In the context of industrial defect detection, edge computing devices have extremely stringent requirements for model inference speed. This project is based on the LiMR model (Lightweight Multi-Scale Detection Model) and achieves real-time performance with an end-to-end inference time of <30ms, throughput >30imgs/s, through deep optimization of the C++ deployment process, which is more than twice as fast as the original implementation. Its core value lies in:

1. Addressing Pain Points in Industrial Deployment: Given the unique nature of defect detection models where both input and output are images, a complete optimization solution for preprocessing → model acceleration → post-processing is provided, filling the gap in similar projects (such as YOLO, MaskRCNN deployment) in edge scenarios.
2. Lowering Development Barriers: Detailed documentation of the installation and configuration of mainstream deep learning C++ libraries (OpenCV, TensorRT, LibTorch, CUDA), as well as the complete process of CMake + VSCode project management, providing developers with reusable project templates.

Technical Architecture and Optimization Solutions

The project has undergone four iterations of optimization, gradually reducing inference time, with key technologies as follows:

1. Preprocessing Acceleration

• Basic OpenCV Version: CPU-based grayscale, normalization, and other operations (time: 11ms).
• OpenCV DNN Acceleration: Utilizing OpenCV’s DNN module to call GPU (time: 8ms).
• OpenCV CUDA + Kernel Function Optimization: Custom CUDA kernel functions for parallel processing (time: 4ms).

• Three Implementation Methods:
• Optimization Effect: Preprocessing stage speedup of 63%, allowing more time for subsequent inference.

• TensorRT Dual Model Acceleration: Converting the model to a TensorRT engine, using FP16 quantization to compress model size and improve computational efficiency.
• FP16 Quantization Optimization: Inference time reduced from 17ms to 8ms, with accuracy loss <1% (acceptable range for industrial scenarios).
• Multi-thread Scheduling: Achieving overlapping data copying and computation through CUDA streams, hiding I/O latency.

• LibTorch Shallow Copy Optimization: Avoiding memory overhead from deep copies, post-processing time reduced from 5ms to 3ms.
• Non-Maximum Suppression (NMS) Parallelization: Utilizing CUDA to accelerate NMS computation, reducing target box filtering time.

• Multi-thread Pipeline Design: Decoding, preprocessing, inference, and post-processing of video streams are split into independent threads, ensuring stable frame rates through lock-free queues.
• Dynamic Batching: Dynamically adjusting the number of images processed in each batch based on device load to maximize GPU utilization.

Key Code Implementation Highlights

1. CMake + VSCode Configuration

   cmake
   # Example: TensorRT library linking configuration
   find_package(CUDA REQUIRED)
   find_package(TensorRT REQUIRED)
   target_link_libraries(your_target PRIVATE ${TensorRT_LIBRARIES} ${CUDA_LIBRARIES})
   

• Cross-Platform Library Management: Unified management of paths and dependencies for libraries like OpenCV and TensorRT through CMake, supporting compilation on both Linux and Windows systems.
• VSCode Debugging Configuration: Integrated CUDA debugging tools, supporting breakpoint debugging and performance analysis.

__global__ void rgb2grayKernel(uchar3* input, uchar* output, int width, int height) {
int x = blockIdx.x * blockDim.x + threadIdx.x;
int y = blockIdx.y * blockDim.y + threadIdx.y;
if (x < width && y < height) {
uchar3 pixel = input[y * width + x];
output[y * width + x] = 0.299f * pixel.x + 0.587f * pixel.y + 0.114f * pixel.z;
}
}

• Custom Grayscale Kernel Function: Parallel processing of each pixel, replacing OpenCV’s serial operations.

// Load serialized engine
std::ifstream engineFile("model.engine", std::ios::binary);
std::vector<char> engineData((std::istreambuf_iterator<char>(engineFile)), std::istreambuf_iterator<char>());
IRuntime* runtime = createInferRuntime(gLogger);
ICudaEngine* engine = runtime->deserializeCudaEngine(engineData.data(), engineData.size(), nullptr);
</char></char></char>

• Serialized Engine Caching: Avoiding repeated engine generation, speeding up startup time.

Performance Comparison and Optimization Effects

• Final Effect: Total time reduced from 33ms to 15ms,120% increase in inference speed, meeting industrial real-time requirements (typically <30ms).

Applicable Scenarios and Scalability

1. Industrial Defect Detection: Suitable for scenarios such as scratches on metal surfaces and defects in electronic components, supporting custom defect types and severity grading.
2. Edge Computing Devices: Can be deployed on low-power devices such as Jetson AGX Xavier and TX2, without relying on cloud computing.
3. Scalability: Supports replacement with other lightweight models (such as YOLOv5s, MobileNetV3), adapting to different input sizes by adjusting preprocessing parameters.

Open Source Value and Significance

1. Filling the Gap in Industrial Deployment: Provides a complete C++ industrial-grade deployment solution, addressing common challenges in model acceleration in edge scenarios.
2. Lowering Technical Barriers: Detailed documentation of the entire process from environment configuration to performance optimization, providing developers with a reference at the level of “copying homework”.
3. Promoting Technological Inclusivity: Through open-source code and tutorials, facilitating the application of computer vision technology in the industrial field.

Real-Time Inference Detection of Industrial Defects (C++)Source Code

https://www.gitpp.com/robolao/project0828009-limr_cpp