Getting Started with Arm-2D: Simplified Porting Guide

Introduction

In our previous article “What is an Embedded System (Part 2) — Deposition Model”, we mentioned:

Current computer technology is approximately 20 years ahead of embedded technology; today’s embedded systems are comparable in resources, theory, and methodology to the cutting-edge computer technology of the 1980s.

GorgonMeducer, WeChat Official Account: Bare Metal Thinking – What is Embedded (Part 2) — “Gravity” and “Deposition”

This means that some technologies applied to personal computers in the early 21st century might gradually be introduced into Deep Embedded Systems — the typical systems that everyone is familiar with are microcontrollers or Cortex-M processors.

Recently, Arm released a 2D graphics acceleration library specifically for all Cortex-M processors on GitHub — Arm-2D:

https://github.com/ARM-software/Arm-2D

According to the description in the GitHub repository’s README, we can simply understand this 2D graphics acceleration library as a standard “graphics driver” specifically for Cortex-M processors. Although the term “graphics driver” is somewhat exaggerated — it seems no Cortex-M processor is “worthy” of such a driver, it is not too far off — because according to the latest trends, as the resources of microcontrollers gradually increase (higher-end process nodes are gradually decreasing in price), processors are not only running faster and having larger memory, but many manufacturers have already considered or are considering equipping Cortex-M processors with dedicated 2D graphics acceleration engines — in this regard, ST and NXP are leading the way — I believe Chrome-Art (also known as DMA2D) is already well-known to everyone. In fact, some chip companies are considering (or have already) equipping the next generation of Cortex-M processors with real 2D-GPUs. In this context, the launch of Arm-2D conveys a clear signal from the industry: Brothers, this is not a drill; an era of dedicated 2D graphics acceleration for Cortex-M has arrived..

What is the Significance of the Graphics Driver for Cortex-M?

Arm is an “old” player in the industry ecosystem — almost all actions are considered from an ecological perspective — so to understand why Arm is customizing a graphics driver for “Cortex-M” at this time, we must understand what has happened in the entire deep embedded ecosystem. To do this, let’s imagine:

You are a product manager at a chip company:

You think smart watch applications should be popular, or IoT terminal devices with color screens should be in demand.
Considering low power consumption and low cost, it seems reasonable for some products to use Cortex-M (instead of Cortex-A) to achieve this. Thus, your company defines a Cortex-M processor equipped with a 2D graphics acceleration engine.
The chip is quickly produced, and the development board is equipped with a 480*272 resolution touchscreen, combined with a specially designed PCB (and a highly recognizable PCB color) — it looks high-end! Impressive! Classy!
But then a problem arises:

There is completely no third-party 2D benchmarking software for microcontrollers (bare metal or RTOS) (the professional term is Benchmark) — how can this demonstrate the power of your hardware? How can evaluation agencies present complex 2D processing in a simple numerical form to the public?
There are so many third-party GUI providers, all of whom have GUI products for Cortex-M chips, but how can I convince them to add support for my chip?

You are a GUI software provider:

Your previous products were somewhat popular in Cortex-A and Linux environments.
Recently, many software companies have targeted the deep embedded market, providing customized GUI products, such as Microsoft’s GUIX, Qt’s Qt for Cortex-M. So you quickly provided corresponding GUI products, but then the problems arose:

There is completely no third-party 2D benchmarking software for microcontrollers……
Unlike the relatively standardized software environment of Cortex-A and Linux, the deep embedded environment is too fragmented:

Different LCD specifications
Different hardware interfaces connecting to LCD (different LCD bandwidth)
Different operating modes of LCD (with or without LCD peripheral drivers, types of drivers)
Different resources of target chips, different system frequencies
Different software environments — countless RTOS, and even bare metal
Every 2D graphics acceleration hardware provided by chip manufacturers is different…

In summary, if you want to support a hardware platform, you need to port and customize it for its hardware…
Considering limited team resources, the officially supported hardware is also limited…

In summary:

The chip manufacturer provides a “customized” 2D graphics acceleration hardware to give the chip a “differentiated” selling point — but from another perspective, it actually exacerbates the fragmentation of hardware. At the same time, chip manufacturers are also struggling to find a large number of GUI protocol stacks to provide software support for their chips;
GUI software providers are struggling with the enormous porting workload caused by hardware fragmentation. They wish their teams could focus more on developing the GUI itself rather than being exhausted by providing “official support” for new hardware platforms.

Suddenly, one day, both parties, exhausted and panting, realize something is off, and they silently turn their heads towards Arm —this so-called neutral third party in the ecosystem…

Then, under everyone’s eager gaze, Arm timidly threw a standard called Arm-2D graphics driver on GitHub, proposing the following resolution:

“How about … I provide an API abstraction layer?”

Chip Manufacturer: Just write the driver for your 2D acceleration engine according to my standards — you will instantly gain support from all GUIs based on Arm-2D;
GUI Provider: Wherever you need hardware acceleration in your GUI, you can directly call the API I provide — if the chip actually has hardware acceleration, it will naturally get accelerated; if the chip does not have hardware acceleration, then use my provided software optimization algorithm — in summary, it’s one port, and it works everywhere, regardless of whether there is actual hardware acceleration.
For bare metal users: Don’t worry, I know your hardware platform resources are tight — you can’t afford a professional GUI protocol stack: you generally either don’t use GUI or just write a simple GUI yourself — don’t worry: Arm-2D also brings you a big gift, allowing you to easily achieve “those unimagined paths”.

“What what?” Bare metal users exclaimed : “Isn’t this a B2B matter? Why suddenly do we have our benefits?”

Stay tuned; I will explain later in the article.

2D Processing Benchmark for Deep Embedded Systems

Although there is no intention to establish third-party 2D benchmarks, Arm-2D provides a reference 2D performance benchmark to demonstrate the 2D processing capabilities of different processors (and different hardware accelerators) under typical GUI loads, with the attitude of “if you really can’t find it, you have to do it yourself” — the official name is: Minimal Frequency Required for 30FPS (MHz).

Its actual principle is as follows:

Establish a typical 2D graphics processing load to simulate the workload and complexity required in daily GUI application scenarios (the static screenshot of this benchmark is approximately the one below; note that this is the effect on the simulator, so the frame rate is very low):

Make different layers float at different speeds and angles to cover more possible scenarios — simulating different complexities that may occur in daily GUIs;
Render 1000 frames, recording the number of CPU clock cycles consumed for each frame, and providing minimum, maximum, and average information.
Using the average cycle count required for frame rendering as a basis, calculate the “minimum processor frequency required for 30FPS” — and use this value as the benchmark result.

It is worth noting that:

This benchmark software does not calculate the time consumed for “sending data from RAM to LCD” when counting the cycles required to render a frame — because the time consumed for “refreshing video memory” is determined by the connection method (or transmission bandwidth) between the chip and the LCD, and is unrelated to the chip’s 2D graphics processing capability.

If you really care about the actual FPS of the system under a certain LCD connection method, just measure the time consumed for “refreshing video memory” (for example, in milliseconds), and then add it to the average milliseconds required for rendering a frame. For example, in the effect diagram shown above, the LCD Latency (the time required to “refresh video memory”) is 43ms, while the average time required to render a frame is 4ms (229FPS). Overall, the actual time consumed for one frame is 43+4 = 47ms (for simplicity, directly calculated as 50ms), at this point, we can consider the final frame rate to be approximately 20FPS (1/50ms).

In fact, it is common for friends to sarcastically say when seeing the dynamic effects of the benchmark: “Hmm, very cool, I just want to know how much capability the processor has left to handle specific applications.” In fact, the “minimum clock frequency required for 30FPS” is intended to intuitively answer this question.

The table above lists several commonly used Cortex-M processors and the minimum frequency required to achieve a refresh rate of 30 frames per second at a resolution of 320*240 (RGB565) purely relying on “processing itself” without using a “dedicated 2D graphics accelerator”, for example:

Cortex-M0+ needs about 194MHz, in other words: if your Cortex-M0+ runs at 250MHz, you still have 56MHz of CPU performance available for applications.

The Raspberry Pi Pico running dual-core Cortex-M0+ at 250MHz is ecstatic

Cortex-M3/M4 requires about 105MHz, meaning if your chip runs at 216MHz, you still have about 111MHz available for specific applications (by the way, which Cortex-M4 doesn’t run at over 100MHz these days?)
Meanwhile, Cortex-M55, which uses SIMD instruction set acceleration (Helium instruction set), only needs 16MHz to achieve a refresh rate of 30FPS —— which is truly terrifying.

More intuitively, here is the official 2D performance multiplier comparison (with Cortex-M4 performance as the baseline). Cortex-M55 Helium is nearly 6 times the performance of Cortex-M4 and 10 times that of Cortex-M0! — If Arm-2D is a graphics driver, then Cortex-M55 Helium is a Cortex-M processor with an integrated “graphics card“.

In terms of deployment, after extensive feedback and testing from the community, we have finally arrived at an era where you can deploy it with just a few clicks — how to deploy Arm-2D in three minutes using CMSIS-Pack is the main content of this article.

Preparation

Prepare an existing project, ensuring that the project can achieve basic LCD initialization and can provide a function for transferring bitmaps to a specified area on the LCD, with the following prototype:

void Disp0_DrawBitmap (uint32_t x,                        uint32_t y,                        uint32_t width,                        uint32_t height,                        const uint8_t *bitmap)

Here, the relationship between the five parameters is shown in the following diagram:

In simple terms, this function copies the pixel information stored in the “continuous storage area” pointed to by the bitmap pointer to a specified rectangular area on the LCD, determined by the position information (x,y) and volume information (width,height).

Many LCDs support a concept called “operating window“; this window is actually the rectangular area shown in the above diagram — once you set the window through instructions, the subsequent continuous pixel writes will be automatically filled into the specified rectangular area (without the user having to consider when to wrap lines).

If you are fortunate enough to use a chip with an LCD controller — the display buffer of the LCD is directly mapped to the 4GB address space of the Cortex-M chip, we can use simple memory read and write operations to implement the above function. For example, with the STM32F746G-Discovery development board:

//! STM32F746G-Discovery#define GLCD_WIDTH     480#define GLCD_HEIGHT    272#define LCD_DB_ADDR   0xC0000000#define LCD_DB_PTR    ((volatile uint16_t *)LCD_DB_ADDR)void Disp0_DrawBitmap (uint32_t x,                          uint32_t y,                          uint32_t width,                          uint32_t height,                          const uint8_t *bitmap) {    volatile uint16_t *phwDes = LCD_DB_PTR + y * GLCD_WIDTH + x;    const uint16_t *phwSrc = (const uint16_t *)bitmap;    for (int_fast16_t i = 0; i &lt; height; i++) {        memcpy ((uint16_t *)phwDes, phwSrc, width * 2);        phwSrc += width;        phwDes += GLCD_WIDTH;    }}

How to Obtain the Installation Package

The first step to deploy Arm-2D in MDK is to obtain the corresponding cmsis-pack. For friends who can smoothly access GitHub, the easiest way is to find the latest .pack file directly through the following URL:

https://github.com/ARM-software/Arm-2D/releases

Of course, you can also find Arm-2D directly in Pack Installer:

For the convenience of domestic users, for friends who cannot access GitHub, you can follow the WeChat official account 【Bare Metal Thinking】 and send the keyword “arm-2d” to obtain the link to the cloud disk.

After obtaining the Arm-2D cmsis-pack, you can directly double-click to install:

If your MDK is a newer version, such as 5.38 or later, you can generally install it by clicking Next without any issues. Otherwise, it is strongly recommended to install the latest MDK to avoid a series of unnecessary troubles — this is also a common suggestion summarized from many predecessors’ experiences in the community — if you start with the latest MDK, you can save a lot of time.

Foolproof Deployment Tutorial

Arm-2D deployment is incredibly simple, and most of the work can be completed by checking the corresponding options in RTE. Please refer to the step-by-step tutorial below:

Step 1: Add Components

In the MDK project, select Project -> Manage -> Run-Time Environment to open the RTE configuration window:

You can also achieve the same goal through the shortcut button in the toolbar:

In the window, find Acceleration and expand it:

As shown in the figure, check the following components:

Core: The core of Arm-2D (required)
Alpha-Blending: Most operations related to transparency, such as mask-based copying, transparent layer composition, transparent color block filling, etc.
Transform: Operations like rotation, scaling, etc. (supports masks, cutouts, and transparency)

To simplify our interface with the LCD display driver, we need to check the following services:

FPB: Partial Frame-Buffer module, the core component supporting partial refreshes
Display Adapter: A code template that uses PFB to adapt to the underlying LCD driver, helping us quickly establish a bridge between upper-level drawing and lower-level LCD refreshing. Generally, the Display Adapter corresponds one-to-one with the screen: if you have one screen, select “1”; if you have two screens, select “2”, and so on.

Since Display Adapter relies on some additional (Extra) modules, if you only check the above parts, you will see an orange warning in the window:

The warning means that Display Adapter relies on the modules Controls and LCD ASCII Printf, but you have not checked them. Simply click the Resolve button in the lower left corner, and RTE will automatically check the dependent modules for you.

Click the “OK” button to complete the addition of components.

For the convenience of subsequent development, it is strongly recommended to download and install the perf_counter module. For specific steps, please refer to the article “【Super Embedded System Performance/Time Toolbox】”; here are the key steps:

Follow the WeChat official account 【Bare Metal Thinking】 and send the keyword “perf_counter” to obtain the corresponding cmsis-pack
Download and install the cmsis-pack
Open RTE, find Utilities, and check Core in perf_counter, recommended to deploy in Source form
If an orange warning appears, click the Resolve button to resolve it

Step 2: Configure the Compilation Environment

If you are using Arm Compiler 6 (armclang), you need to enable support for C11 and GNU extensions, that is, directly select “gnu11” in “Language C”:

If you are using Arm Compiler 5 (armcc), you need to enable support for C99 and GNU extensions, as shown in the following image:

In addition, since Arm-2D relies on CMSIS, you can obtain the latest version of CMSIS by configuring RTE in MDK.

1. As shown in the figure, open the Pack Installer using the button on the far right of the toolbar

We will see a window similar to this:

In the right half of the Packs tab, find ARM::CMSIS, and ensure it shows “Up to date”. If not, click the corresponding button to update. The CMSIS version required for Arm-2D must be at least 5.7.0 (if you are using Cortex-M55, the version must be at least 5.8.0).

2. Using the button in the middle of the toolbar, open the RTE configuration window:

In the Software Component list, expand CMSIS, and check CORE and DSP. It is important to note that if there is a Source option in the DSP section, please select the Source option — this will allow us to directly compile the CMSIS-DSP library in source code form.

In addition, if you are unsure whether the CMSIS used in RTE is the latest version, you can click the Select Packs button:

If the top of the window shows “Use latest Software Packs for Target” is checked, you can basically rest assured. Click OK to close the dialog, and we have successfully added CMSIS to the compilation. Since we chose to compile CMSIS in source code form, we may also need to perform additional settings on the source code of CMSIS-DSP.

At this point, we should be able to successfully complete the compilation.

Step 3: Module Configuration

In the project manager, expand Acceleration, and find the newly added display driver adapter file (arm_2d_disp_adapter_0.h)

Double-click to open it, and select Configuration Wizard in the lower left corner of the editor

Enter the graphical configuration interface:

Fill in the correct information according to your screen:

Color Depth (Screen Colour Depth)
Horizontal Resolution (Width of the screen)
Vertical Resolution (Height of the Screen)
Width of Partial Refresh Buffer Block (Width of the PFB Block), generally consider the width of a full line or half a line of pixels
Height of Partial Refresh Buffer Block (Height of the PFB Block), generally recommended to be 1/10 of the screen pixel height. When RAM is scarce, consider 8 or 1.
When calculating frame rates, how many frames to update data (the Number of iterations), the default is 30, selecting 0 will turn off the real-time frame rate calculation function.
Real-time frame rate statistics mode (FPS Calculation Mode), the default is Render-Only FPS (only calculates the frame rate of the Arm-2D rendering part), while Real FPS calculates the actual frame rate.

Save and close the window.

Find arm_2d_cfg.h in Acceleration,

Similarly, open its Configuration Wizard graphical configuration interface:

Since we are using some modules from Extra, such as Controls and LCD ASCII Printf, we need to provide corresponding information: such as the screen’s color depth, resolution, and the size of the printf buffer (the default value is 64 ASCII characters).

For some options in Arm-2D General Configuration, it is necessary to make some explanations:

Enable Asynchronous Programmers’ model support: Currently recommended to be disabled
Enable anti-alias support for all transform operations: Enables anti-aliasing for rotation and scaling operations
Enable Support for accessing individual Colour channels: Recommended to be enabled when your target screen is RGB888 and you need to support PNG images.

Patches for improving performance includes options to improve performance by disabling certain potentially unused features or functionalities. The recommended practice is:

Do not check
After completing the application, try checking them one by one: If they do not have a significant impact on the application, check them for a performance boost.

Step 4: Add Code

In the source code file where the main() function is located, include the header file:

#include "arm_2d.h"

And complete the initialization of arm-2d in the main() function:

int main(void) {    system_init();     // System initialization including LCD    ...    arm_irq_safe {        arm_2d_init(); // Initialize arm-2d    }    ...    while(1) {        ...    }}

This completes the initialization of the arm-2d module.

If you used the Display Adapter to assist in the porting, you also need to include the following header file:

#include "arm_2d_disp_adapters.h"

And add the following code in the main() function:

int main(void) {    system_init();     // System initialization including LCD    ...    arm_irq_safe {        arm_2d_init(); // Initialize arm-2d    }    // Initialize Display Adapter 0    disp_adapter0_init();        while (true) {        ...        // Execute Display Adapter refresh task        disp_adapter0_task();        ...    } }

Compile and download; if everything goes smoothly, you should see a screen similar to the one below:

The screen displays a continuously rotating “loading circle”, and at the bottom of the screen is the current frame rate information — here, the number after FPS indicates the frame rate of the “drawing” (the number after the colon corresponds to the frame rate time). This separate display method is entirely to help us identify performance bottlenecks.

Thus, we have completed the entire deployment of Arm-2D in a bare metal environment.

How to Start Playing?

I believe many friends are happy but also confused when they see this:

How should I play?

How is this text printed out?

How is the rounded rectangle drawn?

How is the death circle drawn?

What about my own interface?

The content designed here is too much and completely deserves a dedicated article. But before that, please allow me to provide some thoughts and tips for clever friends:

In Acceleration, you can find arm_2d_disp_adapter_0.c, open it to find the key code for using arm-2d
The Display Adapter carries a default scene (Scene), and its drawing function is as follows:

staticIMPL_PFB_ON_DRAW(__pfb_draw_handler){    ARM_2D_UNUSED(pTarget);    ARM_2D_UNUSED(ptTile);    ARM_2D_UNUSED(bIsNewFrame);    arm_2d_canvas(ptTile, __top_container) {            arm_2d_fill_colour(ptTile, NULL, GLCD_COLOR_WHITE);                arm_2d_align_centre(__top_container, 100, 100) {            draw_round_corner_box(  ptTile,                                    &amp;__centre_region,                                    GLCD_COLOR_BLACK,                                    64,                                    bIsNewFrame);        }        busy_wheel2_show(ptTile, bIsNewFrame);    }    arm_2d_op_wait_async(NULL);    return arm_fsm_rt_cpl;}

It is easy to observe that the above two functions actually call Disp0_DrawBitmap() and arm_2d_helper_pfb_report_rendering_complete() before and after:

Common Issues

Issue 1: Installation of CMSIS-Pack failed

If you encounter the following issue when installing cmsis-pack :

Don’t panic; usuallyinstalling the latest MDK and avoiding modifying the default installation directory can resolve this. Friends who want to avoid trouble can follow 【Bare Metal Thinking】 and send the keyword “MDK” to obtain the latest MDK cloud disk link.

It is worth emphasizing that MDK provides a Community version for the open-source community, which, except for commercial use, has almost no restrictions (no restrictions on chips, code size, debugging). The Community version is essentially a type of License, and the installation files are the same as other versions. Friends interested in this “official free version” can obtain it through the following link:

https://www.keil.arm.com/mdk-community/

If you are particularly unlucky and installing the latest MDK does not resolve the above issue, you can try to “fight” by using the import function of Pack-Installer — open Pack Installer , then click File->Import:

Select the downloaded cmsis-pack in the pop-up window for installation.

If it still doesn’t work, please confirm whether your MDK is installed in the default installation directory (C:\Keil_v5); if not, try reinstalling it to the default directory.

If all else fails… just try another computer.

Getting Started with Arm-2D: Simplified Porting Guide

Issue 4: Compiler reports duplicate definitions of certain system intrinsics, such as __set_PRIMASK, etc.

This is due to the CMSIS version included in the project being too low and conflicting with the CMSIS deployed in RTE. For specific solutions, please refer to the article “Guide to the ‘Hellish Achievements’ of CMSIS Players”.

Issue 3: Compiler reports errors related to ARM_PRIVATE()

Such issues arise when there is an independent CMSIS in your MDK project, and this CMSIS conflicts with the CMSIS added in RTE (the CMSIS version in the project is too outdated). For specific solutions, please refer to the article “Guide to the ‘Hellish Achievements’ of CMSIS Players”; I will not elaborate here.

In addition, check whether you have correctly enabled GNU extensions and the corresponding C standards (Arm Compiler 5 requires C99, Arm Compiler 6 requires gnu99)

Issue 4: Prompt cannot find __aeabi_assert

This is due to choosing microLib in the project, and microLib does not provide a low-level implementation for assert.h. Adding the following code will resolve this:

#include "arm_2d.h"#include "cmsis_compiler.h"#if defined(__MICROLIB)void __aeabi_assert(const char *chCond, const char *chLine, int wErrCode) {    ARM_2D_UNUSED(chCond);    ARM_2D_UNUSED(chLine);    ARM_2D_UNUSED(wErrCode);    while(1) {        __NOP();    }}#endif

Issue 4: Prompt cannot find Disp0_DrawBitmap

When you select the Display Adapter service, you need to provide a function for refreshing data to the LCD. When you have multiple screens, you need to select the corresponding number in RTE for the Display Adapter:

At this point, we can see the added code files in Acceleration:

Note that each file here has a corresponding number, indicating the corresponding Display Adapter template. Each Display Adapter needs its own low-level refresh function: Dispn_DrawBitmap(), please refer to the 【Preparation】 section of this article for details.

Issue 5: Hardfault occurs

Check the stack size; it is recommended to be above 0xC00 (3K) for ease.

Conclusion

As long as you have installed the arm-2d cmsis-pack and prepared the LCD low-level driver function Disp0_DrawBitmap() (remember to test it meets the requirements in advance), the entire deployment of Arm-2D can be easily completed in under 3 minutes.

This article seems long because it contains a lot of explanations for “unraveling confusion” and many details to avoid pitfalls for users. Once you successfully follow the tutorial once, the next time you deploy it, it will be much easier — if possible, try to use the latest MDK; this is a recommendation from many predecessors in the community, which can help you avoid 90% of the pitfalls and save a lot of unnecessary time.

The new version of cmsis-pack not only simplifies user deployment but also introduces a service for bare metal users to develop simple GUI applications: Scene Player (scene player) — it allows us to split the interface into several scenes:

Each scene consists of (optional) background and foreground
Users can

Predefine a series of scenes and switch them sequentially
Also, append (Append) new scenes at runtime via API

The scene switching will automatically avoid frame tearing issues

Although arm_2d_disp_adapter_0.c has already demonstrated the use of the Scene Player for us, to lower the learning threshold for everyone, I will detail this “scene-based” low-cost GUI design method in the next article.

Original content is not easy,

If you like my thinking, and find my article enlightening,

Please make sure to “like, bookmark, and share”; it’s very important to me! Thank you!

Welcome to subscribe to Bare Metal Thinking

Related posts

Leave a Comment Cancel reply