0. Introduction
Both Intel’s CPUs and ARM CPUs have SIMD instructions that enable instruction-level parallelism.
This mainly involves knowledge of CPU assembly and some register knowledge. In scenarios involving time-consuming SLAM optimization iterations, such instruction optimizations frequently occur.
SSE is short for Intel’s x86 architecture SIMD instructions, while NEON is short for ARM CPU SIMD instructions.
Recently, while porting SLAM code on the Jetson ARM architecture platform, it was found that it could not be processed quickly, and the GPU itself needed to handle some deep learning model files, thus requiring the use of some parallel acceleration code.
As a learning exercise, I have made some conversions and comparisons of the two SIMD instructions.
1. Basic Definitions
Generally, there is memory in a computer, which acts like a warehouse, while the registers in the CPU represent how much can be taken out at once.
The bit-width of the registers is the same as the instruction width. For example, if we say the instruction width is 128 bits, then the corresponding register width is also 128 bits, and the maximum width of data that the CPU can compute at one time is also 128 bits.
Since the data we commonly use does not reach this width, multiple data calculations can be performed in each instruction cycle. This is what is known as vectorized computation.
For instance, a floating-point number occupies 4 bytes or 8 bytes, while integers can occupy 1, 2, or 4 bytes depending on the application, thus the vectorization acceleration also varies.
Therefore, one instruction can complete a maximum of 4 floating-point calculations, while it can compute 16 int8_t data.
2. Comparison of NENO and SSE Instructions
Tip: Here, f represents floating-point numbers, p represents pointers, i represents integers, and m128 represents an array of float32x4.
3. Optimization Techniques
-
Pay attention to the order of instructions. Why? Because the CPU works in a pipeline, the execution time of adjacent instructions does not necessarily start after one instruction has finished executing. However, once data dependencies are encountered, blocking may occur. If we arrange the order of instructions well to minimize data dependencies, or when they do occur, the previous instruction has already completed. Thus, slightly modifying the execution order of instructions can speed up the code. Isn’t that fascinating?
-
Appropriate loop unrolling; note that it should only be appropriate, as the goal is still to reduce data dependencies, but unrolling too much may lead to criticism.
4. Related Official Links
-
NEON instruction authoritative official website:
https://developer.arm.com/architectures/instruction-sets/simd-isas/neon/intrinsics
-
NEON instruction Chinese quick reference:
https://blog.csdn.net/billbliss/article/details/78924636
-
Summary of NEON instructions, refer to this article:
https://blog.csdn.net/xiongtiancheng/article/details/77103810
-
Development related to SSE:
https://www.wanweibaike.net/wiki-SSE?new
-
Summary of SSE instructions, refer to this article:
https://blog.csdn.net/qq_27825451/article/details/103934359
5. NENO Example
Vector Multiplication:
The following operation multiplies each dimension of the vector and then sums them up, which has good parallelism, so it can be accelerated using ARM NEON intrinsic instructions. Below is the code implementation:
inline float dot(const float* A, const float* B, int K){ float sum = 0; float32x4_t sum_vec = vdupq_n_f32(0);//, left_vec, right_vec; for (int k = 0; k < K; k += 8) { sum_vec = vmlap_f32(sum_vec, vld1q_f32(A + k), vld1q_f32(B + k)); sum_vec = vmlap_f32(sum_vec, vld1q_f32(A + k + 4), vld1q_f32(B + k + 4)); // sum_vec = vmlap_f32(sum_vec, vld1q_f32(A + k+8), vld1q_f32(B+k+8)); // sum_vec = vmlap_f32(sum_vec, vld1q_f32(A + k+12), vld1q_f32(B+k+12)); } float32x2_t r = vadd_f32(vget_high_f32(sum_vec), vget_low_f32(sum_vec)); sum += vget_lane_f32(vpadd_f32(r, r), 0); return sum; }Exp Acceleration:
The basic principle of the algorithm is ingeniously designed considering the layout of float data types in memory. To understand more details, you can refer to the original blog. This article only introduces how to accelerate it using ARM NEON intrinsic instructions (compared to the original blog, the second constant in the code has changed slightly; this new constant is what I experimented with, resulting in less error).
The advantage of ARM NEON intrinsic instructions is parallel computation, so we perform exp on each element of an array and sum them, then accelerate it:
inline float expf_sum(float* score,int len){ float sum=0; float32x4_t sum_vec=vdupq_n_f32(0); float32x4_t ai=vdupq_n_f32(1064807160.56887296), bi; int32x4_t int_vec; int value; for(int i=0;i < len;i+=4) { bi=vld1q_f32(score+4*i); sum_vec=vmlaq_n_f32(ai,bi,12102203.1616540672); int_vec=vcvtq_s32_f32(sum_vec); value=vgetq_lane_s32(int_vec,0); sum+=(*(float*)(&value)); value=vgetq_lane_s32(int_vec,1); sum+=(*(float*)(&value)); value=vgetq_lane_s32(int_vec,2); sum+=(*(float*)(&value)); value=vgetq_lane_s32(int_vec,3); sum+=(*(float*)(&value)); } return sum;}
6. Reference Links
-
https://x007dwd.github.io/2017/03/18/SSE-vs-NEON/
-
https://blog.csdn.net/qq_20880415/article/details/101290987
-
https://blog.csdn.net/qq_20880415/article/details/100050059
-
https://aijishu.com/a/1060000000217479
-
https://github.com/intel/ARM_NEON_2_x86_SSE
-
https://github.com/jratcliff63367/sse2neon
-
https://blog.csdn.net/jgj123321/article/details/95633431
-
https://blog.csdn.net/qq_27825451/article/details/103934359
-
https://blog.csdn.net/weixin_40593924/article/details/104972350
“Matlab Robot Toolbox Introduction Guide”
As an introductory course for the Robot Toolbox, it will gradually introduce the installation of the toolbox, the differences and conversions of various pose transformation methods, robot modeling under the D-H method, motion planning in joint space, and spatial pose interpolation.
(Scan to view course details)
Click “Read the original text” to join the learning.