ARMv8 Memory System Study Notes

Cache coherency

Cacheability

Normal memory can be set as cacheable or non-cacheable, and can be set separately for inner and outer.

Shareability

If set to non-shareable, that segment of memory is only used by a specific core. If set to inner shareable or outer shareable, it can be accessed by other observers (other cores, GPU, DMA devices). The distinction between inner and outer is the range of cache coherence required, and the division between inner and outer observers is implementation defined.

PoC & PoU

When cleaning or invalidating the cache, it can operate at a specific cache level, specifically to the following two “points”:Point of coherency (PoC): Ensures that all observers that can access memory (CPU cores, DSP, DMA devices) see the same copy of a memory address at this “point,” generally the main memory.Point of unification (PoU): Ensures that a core’s icache, dcache, MMU (TLB) sees the same copy of a memory address at this “point.” For example, the unified L2 cache is the PoU of the core in the image below; if there is no L2 cache, then it is the main memory.When saying “invalidate icache to PoU,” it means to invalidate the icache so that the next access reads from the L2 cache (PoU).An application scenario for PoU is: after modifying its own code during execution, using two steps to refresh the cache, first, clean the dcache to PoU, then invalidate the icache to PoU.

Memory consistency

ARMv8-A adopts a weak memory model, where reads and writes to normal memory may be executed out of order. The page table can be configured for non-reordering (which can be used for device memory).Normal memory: RAM, Flash, ROM in physical memory, these types of memory can be accessed in a weak memory order to improve performance.In a single-core single-thread environment, consecutive dependent str and ldr will not be affected by weak memory order, for example:

str x0, [x2]ldr x1, [x2]

Barriers

ISB

Flushes the current PE’s pipeline, requiring subsequent instructions to be re-read from cache or memory, and ensures that subsequent instructions can see the context-changing operations that occurred before this instruction, specifically, including modifications to ASID, TLB maintenance instructions, and modifications to any system registers.

DMB

Ensures that other observers within the specified shareability domain see data accesses that occur after a dmb before they see data accesses that occurred before the dmb:

str x0, [x1]dmbstr x2, [x3] // If an observer sees this line str, it must also see the first line str

Additionally, dmb ensures that all data access instructions following it can see the dcache or unified cache maintenance operations that occurred before it:

dc csw, x5ldr x0, [x1] // May not see dcache clean<code>

dmb ishldr x2, [x3] // Must see dcache clean

DSB

Ensures the same memory order as dmb, but in addition to memory access operations, it also ensures that any subsequent instructions can see the results of previous data accesses.Waits for all cache, TLB, and branch prediction maintenance operations initiated by the current PE to be visible to the specified shareability domain.Can be used to ensure data synchronization before the sev instruction.An example:

str x0, [x1] // update a translation table entrydsb ishst // ensure write has completedtlbi vae1is, x2 // invalidate the TLB entry for the entry that changesdsb ish // ensure that TLB invalidation is completeisb // synchronize context on this processor

DMB & DSB options

dmb and dsb can specify the type of memory operation constraints and shareability domain through options:

One-way barriers

Load-Acquire (LDAR): All loads and stores that are after an LDAR in program order, and that match the shareability domain of the target address, must be observed after the LDAR.
Store-Release (STLR): All loads and stores preceding an STLR that match the shareability domain of the target address must be observed before the STLR.
LDAXR
STLXR

Unlike the data barrier instructions, which take a qualifier to control which shareability domains see the effect of the barrier, the LDAR and STLR instructions use the attribute of the address accessed.

C++ & Rust memory order

Relaxed

Relaxed atomic operations only guarantee atomicity, not synchronization semantics.

// Thread 1:r1 = y.load(std::memory_order_relaxed); // Ax.store(r1, std::memory_order_relaxed); // B// Thread 2:r2 = x.load(std::memory_order_relaxed); // Cy.store(42, std::memory_order_relaxed); // D

The above code, when compiled on ARM, uses str and ldr instructions, which may be executed out of order, potentially resulting in r1 == r2 == 42, meaning A sees D, and C sees B.A typical use case for relaxed ordering is simply incrementing a counter, such as the reference count in std::shared_ptr, where only atomicity needs to be guaranteed, with no memory order requirements.

Release-acquire

Rel-acq atomic operations, in addition to guaranteeing atomicity, also ensure synchronization between stores using release and loads using acquire; acquire must see the instructions before release, and release must not see the instructions after acquire.

#include &lt;thread&gt;#include &lt;atomic&gt;#include &lt;cassert&gt;#include &lt;string&gt;std::atomic&lt;std::string *&gt; ptr;int data;void producer() {    std::string *p = new std::string("Hello");    data = 42;    ptr.store(p, std::memory_order_release);}void consumer() {    std::string *p2;    while (!(p2 = ptr.load(std::memory_order_acquire)))        ;    assert(*p2 == "Hello"); // never fires    assert(data == 42); // never fires}int main() {    std::thread t1(producer);    std::thread t2(consumer);    t1.join(); t2.join();}

The above code, once the consumer successfully loads a non-null string pointer from ptr, it must see the write operation data = 42.

This code compiles on ARM to use stlr and ldar, but the semantics defined by C++ are actually weaker than what stlr and ldar provide; C++ only guarantees synchronization between the two threads using release and acquire.A typical use case for rel-acq ordering is mutex or spinlock, where when releasing a lock, all memory accesses in the critical section before the release must be visible to observers acquiring the lock simultaneously.

Release-consume

Similar to rel-acq, but does not guarantee that memory accesses after consume do not complete before release; it only guarantees that instructions dependent on the consume load operation are not executed early, meaning that after consume is not a critical section, but just uses the results of memory accesses before release.

Note that currently (2/2015) no known production compilers track dependency chains: consume operations are lifted to acquire operations.

#include &lt;thread&gt;#include &lt;atomic&gt;#include &lt;cassert&gt;#include &lt;string&gt;std::atomic&lt;std::string *&gt; ptr;int data;void producer() {    std::string *p = new std::string("Hello");    data = 42;    ptr.store(p, std::memory_order_release);}void consumer() {    std::string *p2;    while (!(p2 = ptr.load(std::memory_order_consume)))        ;    assert(*p2 == "Hello"); // never fires: *p2 carries dependency from ptr    assert(data == 42); // may or may not fire: data does not carry dependency from ptr}int main() {    std::thread t1(producer);    std::thread t2(consumer);    t1.join(); t2.join();}

In the above code, since assert(data == 42) does not depend on the consume load instruction, it may execute before loading a non-null pointer, which means it does not guarantee seeing the release store, and thus does not guarantee seeing data = 42.

Sequentially-consistent

Seq-cst ordering guarantees a similar memory order to rel-acq; if a thread’s seq-cst load sees another thread’s seq-cst store, it must see the instructions before the store, and instructions after the load will not see instructions before the store. Additionally, seq-cst guarantees that all seq-cst instructions seen by each thread have a consistent total order.A typical use case is in scenarios with multiple producers and multiple consumers, ensuring that multiple consumers see a consistent total order of producer operations.

#include &lt;thread&gt;#include &lt;atomic&gt;#include &lt;cassert&gt;std::atomic&lt;bool&gt; x = {false};std::atomic&lt;bool&gt; y = {false};std::atomic&lt;int&gt; z = {0};void write_x() {    x.store(true, std::memory_order_seq_cst);}void write_y() {    y.store(true, std::memory_order_seq_cst);}void read_x_then_y() {    while (!x.load(std::memory_order_seq_cst))        ;    if (y.load(std::memory_order_seq_cst)) {        ++z;    }}void read_y_then_x() {    while (!y.load(std::memory_order_seq_cst))        ;    if (x.load(std::memory_order_seq_cst)) {        ++z;    }}int main() {    std::thread a(write_x);    std::thread b(write_y);    std::thread c(read_x_then_y);    std::thread d(read_y_then_x);    a.join(); b.join(); c.join(); d.join();    assert(z.load() != 0); // will never happen}

In the above code, read_x_then_y and read_y_then_x cannot see opposite assignments of x and y, so at least one must execute ++z.Seq-cst and other ordering mixed may lead to unexpected results, as in the following example, for thread 1, A sequenced before B, but for other threads, they may see B first and A later, hence C may see B and get r1 = 1, D sees E and gets r2 = 3, F does not see A and gets r3 = 0.

// Thread 1:x.store(1, std::memory_order_seq_cst); // Ay.store(1, std::memory_order_release); // B// Thread 2:r1 = y.fetch_add(1, std::memory_order_seq_cst); // Cr2 = y.load(std::memory_order_relaxed); // D// Thread 3:y.store(3, std::memory_order_seq_cst); // Er3 = x.load(std::memory_order_seq_cst); // F

References

https://developer.arm.com/documentation/100941/0100
https://developer.arm.com/documentation/den0024/a/Caches
https://developer.arm.com/documentation/den0024/a/Memory-Ordering
https://developer.arm.com/documentation/dui0489/c/arm-and-thumb-instructions/miscellaneous-instructions/dmb–dsb–and-isb
https://en.cppreference.com/w/cpp/atomic/memory_order