When Will Go Officially Support SIMD?

Single Instruction Multiple Data (SIMD) is a parallel computing technology that allows a single instruction to process multiple data points simultaneously. SIMD is widely used in modern CPUs and can significantly enhance the performance of compute-intensive tasks such as image processing, machine learning, and scientific computing. As the application of the Go language in high-performance computing increases, SIMD support has become a focal point of interest for developers.

Currently, many mainstream and emerging languages have corresponding simd libraries, such as C++, Rust, Zig, etc., but the official support for simd in Go has been under discussion (issue#67520^[1]). The design goals of Go are simplicity and portability, while the implementation of SIMD often requires optimization for different hardware architectures, which conflicts with Go’s design goals. Therefore, the support for SIMD in Go has been a topic of controversy. Recently, discussions around this issue have become more active, and there is hope for quicker support.

1. Background of Go Language and SIMD

1.1 Performance Pursuit of Go Language

The Go language has gained widespread application due to its concise syntax, efficient concurrency model, and fast compilation speed. However, Go has faced challenges in performance optimization, especially in scenarios that require processing large amounts of data. SIMD, as an efficient parallel computing technology, can significantly enhance computational performance, which has led to increasing calls for SIMD support from the Go community.

Without SIMD, we would miss many potential optimizations. Below is a non-exhaustive list of specific items that can improve performance in everyday scenarios:

simdjson^[2]
Decoding billions of integers per second through vectorization^[3]
Vectorized and performance-portable quicksort^[4]
Introduction to Hyperscan^[5]
From slow to SIMD: A Go optimization story^[6]
How to Use AVX512 in Golang via C Compiler^[7]

Additionally, it will make the currently existing packages more portable and maintainable:

simdjson-go^[8]
SHA256-SIMD^[9]
MD5-SIMD^[10]

In the upcoming Go 1.24 release, the built-in map will be replaced with Swiss Tables, which uses SIMD code^[11] optimized for the AMD64 architecture. Is this the first time SIMD instructions have been introduced into the Go official codebase?

Previously, there were also attempts to implement SIMD-accelerated encoding/hex, which were rejected (^[12]), and the reasons were quite valid: the acceleration effect was good, but it seemed too complex and violated Go’s principle of simplicity. Similar cases includeunicode/utf8: make Valid use AVX2 on amd64^[13]

In fact, the Go official team has already used SIMD instructions in the standard library crypto/sha256 in 2023 (crypto/sha256: add sha-ni implementation)^[14].

1.2 Basic Concepts of SIMD

SIMD processes multiple data points simultaneously through a single instruction, typically used for vectorized computations. Modern CPUs (such as Intel’s SSE/AVX and ARM’s NEON) provide SIMD instruction sets that allow developers to accelerate computational tasks through specific instruction sets. However, directly using SIMD instruction sets often requires writing assembly code or using specific compiler built-in functions, which poses higher demands on developers.

1.2.1 Core Idea of SIMD

The core idea of SIMD is to process multiple data points simultaneously through a single instruction. For example, traditional scalar addition instructions can only handle two numbers at a time, while SIMD addition instructions can handle multiple numbers (such as 4, 8, or even more) simultaneously. This parallel processing method can significantly enhance the performance of compute-intensive tasks.

1.2.2 Composition of SIMD Instruction Sets

SIMD instruction sets typically include the following categories of instructions:

Arithmetic Operations: Addition, subtraction, multiplication, division, etc.
Logical Operations: AND, OR, NOT, XOR, etc.
Data Movement: Loading, storing, rearranging data.
Comparison Operations: Comparing multiple data points and generating masks.
Special Operations: Such as square root, absolute value, maximum, minimum, etc.

1.3 Common Instruction Sets

1.3.1 Intel’s SIMD Instruction Sets

1.3.1.1 MMX (MultiMedia eXtensions)

Release Date: 1996
Register Width: 64 bits
Data Types: Integers (8-bit, 16-bit, 32-bit)
Features:

Mainly used for multimedia processing.
Introduced 8 64-bit registers (MM0-MM7).
Does not support floating-point operations.

1.3.1.2 SSE (Streaming SIMD Extensions)

Release Date: 1999
Register Width: 128 bits
Data Types: Single-precision floating-point (32 bits), integers (8-bit, 16-bit, 32-bit, 64-bit)
Features:

Introduced 8 128-bit registers (XMM0-XMM7).
Supports floating-point operations, suitable for scientific computing and graphics processing.
Subsequent versions (SSE2, SSE3, SSSE3, SSE4) added more instructions and features.

1.3.1.3 AVX (Advanced Vector Extensions)

Release Date: 2011
Register Width: 256 bits
Data Types: Single-precision floating-point (32 bits), double-precision floating-point (64 bits), integers (8-bit, 16-bit, 32-bit, 64-bit)
Features:

Introduced 16 256-bit registers (YMM0-YMM15).
Supports wider vector operations, further enhancing performance.
Subsequent versions (AVX2, AVX-512) support more complex operations and wider registers (512 bits).

1.3.1.4 AVX-512

Release Date: 2016
Register Width: 512 bits
Data Types: Single-precision floating-point (32 bits), double-precision floating-point (64 bits), integers (8-bit, 16-bit, 32-bit, 64-bit)
Features:

Introduced 32 512-bit registers (ZMM0-ZMM31).
Supports more complex operations, such as masked operations and broadcast operations.
Mainly used in high-performance computing and artificial intelligence fields.

1.3.2 ARM’s SIMD Instruction Sets

1.3.2.1 NEON

Release Date: 2005
Register Width: 128 bits
Data Types: Single-precision floating-point (32 bits), integers (8-bit, 16-bit, 32-bit, 64-bit)
Features:

Widely used in mobile devices and embedded systems.
Supports 16 128-bit registers (Q0-Q15).
Suitable for multimedia processing, signal processing, etc.

1.3.2.2 SVE (Scalable Vector Extension)

Release Date: 2016
Register Width: Variable (128 bits to 2048 bits)
Data Types: Single-precision floating-point (32 bits), double-precision floating-point (64 bits), integers (8-bit, 16-bit, 32-bit, 64-bit)
Features:

Supports variable-length vector operations to adapt to different hardware configurations.
Introduces predicate registers, supporting conditional execution.
Mainly used in high-performance computing and machine learning.

1.4 Compiler Built-in Functions

Most modern compilers (such as GCC, Clang, MSVC) provide built-in functions for SIMD instruction sets, allowing developers to call SIMD instructions without writing assembly code.

1.5 Automatic Vectorization

Some compilers support automatic vectorization, which can automatically convert scalar code into SIMD code. For example, when compiling the following code with GCC, automatic vectorization can be enabled:

gcc -O3 -mavx2 -o program program.c

2. Current Status of SIMD Support in Go Language

2.1 SIMD Support in Go Language Standard Library

The Go language standard library does not yet provide direct support for SIMD. The Go compiler (gc) also lacks automatic vectorization capabilities, meaning developers cannot automatically generate SIMD code via the compiler as they can in C/C++.

In Issue #67520^[15], discussions are still dragging on, often deviating to the specifics of implementation (build tag).

2.2 Third-party Libraries and Solutions

Despite the lack of direct support for SIMD in the Go language standard library, the community has developed several third-party libraries and tools to help developers use SIMD instruction sets in Go. In#67520^[16], Clement Jean also provided a conceptual implementation proposal: simd-go-POC^[17].

Below are some third-party implementations of SIMD instructions (not libraries based on SIMD like sonic, simdjson-go, etc.):

2.2.1 `kelindar/simd`

The kelindar/simd library contains a set of vectorized mathematical functions that use the clang compiler for automatic vectorization and convert to Go’s PLAN9 assembly code. For CPUs that do not support vectorization, or for CPUs for which this library has not generated code, a generic version is also provided.

Currently, it only supports AVX2, but generating code for AVX512 and SVE (for ARM) should be relatively easy. Most of the code in this library is automatically generated, which helps with maintenance.

sum := simd.SumFloat32s([]float32{1, 2, 3, 4, 5})

2.2.2 `alivanz/go-simd`

[alivanz/go-simd](https://github.com/alivanz/go-simd) implements SIMD (Single Instruction Multiple Data) operations in Go, specifically optimized for the ARM NEON architecture. Its goal is to provide optimized parallel processing capabilities for specific computational tasks. Below is an example of addition and multiplication:

package main

import (
"log"

"github.com/alivanz/go-simd/arm"
"github.com/alivanz/go-simd/arm/neon"
)

func main() {
var a, b arm.Int8X8
var add, mul arm.Int16X8
for i := 0; i < 8; i++ {
  a[i] = arm.Int8(i)
  b[i] = arm.Int8(i * i)
 }
 log.Printf("a = %+v", b)
 log.Printf("b = %+v", a)
 neon.VaddlS8(&add, &a, &b)
 neon.VmullS8(&mul, &a, &b)
 log.Printf("add = %+v", add)
 log.Printf("mul = %+v", mul)
}

2.2.3 `pehringer/simd`

The pehringer/simd library provides SIMD support through Go assembly, implementing arithmetic operations, bitwise operations, and max/min operations. It allows parallel element-wise calculations, resulting in speed improvements of 100% to 400%. Currently, it supports AMD64 (x86_64) and ARM64 processors.

2.3 Go Assembly and SIMD

The Go language supports calling CPU instruction sets directly through assembly code, which makes the implementation of SIMD possible. Developers can write Go assembly code to call specific SIMD instruction sets (such as SSE, AVX, etc.) to achieve high-performance vectorized computations. However, writing and maintaining assembly code poses higher demands on developers and has poorer portability.

// Below is a simple Go assembly example using AVX instruction set for vector addition
TEXT ·add(SB), $0-32
    MOVQ a+0(FP), DI
    MOVQ b+8(FP), SI
    MOVQ result+16(FP), DX
    MOVQ len+24(FP), CX

    TESTQ CX, CX        ; Check if length is 0
    JZ done             ; Return directly if 0

    MOVQ CX, R8         ; Save original length
    SHRQ $2, CX         ; Divide by 4 to get loop count
    JZ remainder        ; If less than 4 elements, jump to handle remainder

    XORQ R9, R9         ; Index counter starting from 0
loop:
    VMOVUPD (DI)(R9*8), Y0
    VMOVUPD (SI)(R9*8), Y1
    VADDPD Y0, Y1, Y0
    VMOVUPD Y0, (DX)(R9*8)
    ADDQ $4, R9
    DECQ CX
    JNZ loop

remainder:              ; Handle remaining elements
    ANDQ $3, R8        ; Get remainder
    JZ done
    ; Add code to handle remainder here

done:
    RET

Of course, the addresses of arrays a, b, and result need to be aligned for optimal performance.

Conclusion

Although the Go language currently lacks comprehensive support for SIMD, the community has provided some solutions through third-party libraries and assembly code. In the future, with improvements to the Go compiler and standard library support (it is believed that Go will eventually support it), the potential of the Go language in high-performance computing will be further unlocked. For developers, mastering SIMD technology will help write more efficient Go code to tackle increasingly complex computational tasks.

References[1]

issue#67520: https://github.com/golang/go/issues/67520

[2]

simdjson: https://github.com/simdjson/simdjson

[3]

Decoding billions of integers per second through vectorization: https://people.csail.mit.edu/jshun/6886-s19/lectures/lecture19-1.pdf

[4]

Vectorized and performance-portable quicksort: https://opensource.googleblog.com/2022/06/Vectorized%20and%20performance%20portable%20Quicksort.html

[5]

Introduction to Hyperscan: https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-hyperscan.html

[6]

From slow to SIMD: A Go optimization story: https://sourcegraph.com/blog/slow-to-simd

[7]

How to Use AVX512 in Golang via C Compiler: https://gorse.io/posts/avx512-in-golang.html#convert-assembly

[8]

simdjson-go: https://github.com/minio/simdjson-go

[9]

SHA256-SIMD: https://github.com/minio/sha256-simd

[10]

MD5-SIMD: https://github.com/minio/md5-simd

[11]

SIMD code: https://go-review.googlesource.com/c/go/+/626277

[12]

Rejected: https://go-review.googlesource.com/c/go/+/110195

[13]

unicode/utf8: make Valid use AVX2 on amd64: https://go-review.googlesource.com/c/go/+/535838

[14]

crypto/sha256: add sha-ni implementation: https://go-review.googlesource.com/c/go/+/408795

[15]

#67520: https://github.com/golang/go/issues/67520

[16]

#67520: https://github.com/golang/go/issues/67520

[17]

simd-go-POC: https://github.com/Clement-Jean/simd-go-POC

[18]

kelindar/simd: https://github.com/kelindar/simd

[19]

pehringer/simd: https://github.com/pehringer/simd

When Will Go Officially Support SIMD?

1. Background of Go Language and SIMD

1.1 Performance Pursuit of Go Language

1.2 Basic Concepts of SIMD

1.2.1 Core Idea of SIMD

1.2.2 Composition of SIMD Instruction Sets

1.3 Common Instruction Sets

1.3.1 Intel’s SIMD Instruction Sets

1.3.1.1 MMX (MultiMedia eXtensions)

1.3.1.2 SSE (Streaming SIMD Extensions)

1.3.1.3 AVX (Advanced Vector Extensions)

1.3.1.4 AVX-512

1.3.2 ARM’s SIMD Instruction Sets

1.3.2.1 NEON

1.3.2.2 SVE (Scalable Vector Extension)

1.4 Compiler Built-in Functions

1.5 Automatic Vectorization

2. Current Status of SIMD Support in Go Language

2.1 SIMD Support in Go Language Standard Library

2.2 Third-party Libraries and Solutions

2.2.1 `<span>kelindar/simd</span>`

2.2.2 `<span>alivanz/go-simd</span>`

2.2.3 `<span>pehringer/simd</span>`

2.3 Go Assembly and SIMD

Conclusion

Leave a Comment Cancel reply

1. Background of Go Language and SIMD

1.1 Performance Pursuit of Go Language

1.2 Basic Concepts of SIMD

1.2.1 Core Idea of SIMD

1.2.2 Composition of SIMD Instruction Sets

1.3 Common Instruction Sets

1.3.1 Intel’s SIMD Instruction Sets

1.3.1.1 MMX (MultiMedia eXtensions)

1.3.1.2 SSE (Streaming SIMD Extensions)

1.3.1.3 AVX (Advanced Vector Extensions)

1.3.1.4 AVX-512

1.3.2 ARM’s SIMD Instruction Sets

1.3.2.1 NEON

1.3.2.2 SVE (Scalable Vector Extension)

1.4 Compiler Built-in Functions

1.5 Automatic Vectorization

2. Current Status of SIMD Support in Go Language

2.1 SIMD Support in Go Language Standard Library

2.2 Third-party Libraries and Solutions

2.2.1 <span>kelindar/simd</span>

2.2.2 <span>alivanz/go-simd</span>

2.2.3 <span>pehringer/simd</span>

2.3 Go Assembly and SIMD

Conclusion

Related posts

Leave a Comment Cancel reply

2.2.1 `<span>kelindar/simd</span>`

2.2.2 `<span>alivanz/go-simd</span>`

2.2.3 `<span>pehringer/simd</span>`