Understanding SIMD Instructions

Author | SatanWoo

SIMD is a common computing method that processes multiple data points with a single instruction. This article serves as an introduction to SIMD, starting with a simple understanding of its usage and concepts.

Meaning of SIMD

SIMD stands for Single Instruction Multiple Data. In brief, it refers to the ability to process multiple data points with a single instruction. We know that a program consists of a series of machine instructions, but executing a single machine code actually involves multiple processes, including fetching the instruction, decoding it, and executing it, as illustrated below (for now, we will ignore pipelining and parallelism).

At each stage, one or more machine cycles are consumed. If we assume that fetching and decoding instructions can be completed in approximately one machine cycle, then different instructions can consume significantly different machine cycles during the execution phase.

For example, a simple addition instruction may require two machine cycles to execute, while multiplication may require 5-6 machine cycles. Therefore, when we cannot shorten the execution cycle of an instruction, using SIMD technology allows us to process more data within the same execution cycle, which also increases the data throughput per unit time and enhances computational performance.

The Intel manual provides a series of parallel instructions, including MMX, SSE, AVX, etc., targeting different lengths of data in parallel, such as:

• MMX for parallel computation of 64-bit data.

• SSE for parallel computation of 128-bit data.

• AVX for parallel computation of 256-bit data.

• AVX512 for parallel computation of 512-bit data.

For more detailed usage, please refer to:

Intel Manual

How to Use SIMD

Since most people are not very familiar with SIMD, this article will use a familiar environment, Xcode + x86/64 architecture, to demonstrate.

We will rewrite a simple 256-bit (32 byte) addition in SIMD form to verify:

Original version:

double input1[k] = {1, 2, 3, 4};
double input2[k] = {5, 6, 7, 8};
double result[k] = {0};

for (int i = 0; i < k; i++) {
    result[i] = input1[i] + input2[i];
}

SIMD version:

const int k = 4;
double input1[k] = {1, 2, 3, 4};
double input2[k] = {5, 6, 7, 8};
double result[k] = {0};

__m256d a = _mm256_load_pd(input1);
__m256d b = _mm256_load_pd(input2);

__m256d c = _mm256_add_pd(a, b);
_mm256_store_pd(result, c);

The original version is easier to understand. Let’s take a closer look at what the SIMD code means:

• _mm256_load_pd reads from a memory address, returning a __m256d vector (256-bit). The definition of __m256d is as follows:

typedef double __m256d __attribute__((__vector_size__(32)));

This means that the length of __m256d is 32 bytes (256 bits), consisting of 4 double elements.

• _mm256_add_pd directly adds two 256-bit vector elements together.

• _mm256_store_pd is the inverse operation of _mm256_load_pd, which will not be elaborated here.

Note: If prompted for AVX support, please add the Compiler Flag: -mavx in the corresponding code file in Xcode.

Using SIMD for Summation

Since the essence of SIMD is to improve computational throughput per unit time, let’s practice with a simple example of summation:

The conventional code is as follows:

double CommonAdd(double *data, int count)
{
    double result = 0;

    for (int i = 0; i < count; i++) {
        result += data[i];
    }

    return result;
}

The SIMD code is as follows:

double AVXAdd(double *data, int count)
{
    int offset = 0;

    __m256d v1;
    __m256d sum = _mm256_setzero_pd();

    double ret = 0;

    for (int i = 0; i < count/4; i++) {
        v1 = _mm256_load_pd(data + offset);
        sum = _mm256_add_pd(sum, v1);
        offset += 4;
    }

    sum = _mm256_hadd_pd(sum, sum); // Horizontal sum

    ret += sum[0];
    ret += sum[2];

    return ret;
}

The test code is as follows:

int main() {

    struct  timeval   start;
    struct  timeval   end;

    const int k = 512 * 512;

    const int loop = 1;

    double input1[k];

    for (int i = 0; i < k; i++) {
        input1[i] = i;
    }

    gettimeofday(&start, nullptr);

    for (int j = 0; j < loop; j++) {
        CommonAdd(input1, k);
    }

    gettimeofday(&end, nullptr);

    printf("tv_sec:%ld\n",end.tv_sec - start.tv_sec);
    printf("tv_usec:%d\n", end.tv_usec - start.tv_usec);

    std::cout << " ======================= " << std::endl;

    gettimeofday(&start, nullptr);

    for (int j = 0; j < loop; j++) {
        AVXAdd(input1, k);
    }

    gettimeofday(&end, nullptr);

    printf("tv_sec:%ld\n",end.tv_sec - start.tv_sec);
    printf("tv_usec:%d\n", end.tv_usec - start.tv_usec);

    return 0;
}

Here, we chose a common size of 512 * 512 in image processing for verification. On my 2015 MacBook Pro, I obtained the following performance timings:

• Conventional method 【774 us】

• SIMD 【560 us】

Do not underestimate this performance difference; it can have a significant impact on large computational tasks in edge-side deep learning.

Postscript

This article only introduces the most conventional usage of SIMD. However, in actual design processes, it is not possible to apply it as simply as we have. Along the way, you will encounter many different pitfalls, including improper applications that lead to performance degradation and crash issues. These will be addressed in future discussions.

Recommended Reading• The Ultimate Guide to Correctly Using Animations• These iOS Libraries Are Great• Flutter v1.12 New Features: Integration into Existing Applications• Some Personal Views on Interviews• Discussing Optimizations for Autorelease