Detailed Analysis of the U-Boot Boot Process on ARMv8 Architecture

This article is based on the ARMv8 architecture to analyze the U-Boot boot process, with U-Boot version 2022-01.

1 Overview

First, let’s quote an introduction from Wiki: U-Boot is a bootloader primarily used for embedded systems that can support various computer system architectures. U-Boot was initially developed by the DENX Software Engineering team in Germany, and many embedded developers interested in open-source bootloader porting have continuously expanded and deepened the porting work of various series of embedded processors, supporting more embedded operating systems for loading and booting.

Reasons for choosing U-Boot:

Open source; supports booting various embedded operating system kernels such as Linux, NetBSD, VxWorks, QNX, RTEMS, ARTOS, LynxOS, Android; supports multiple processor series such as PowerPC, ARM, x86, MIPS; high reliability and stability; highly flexible functional settings suitable for U-Boot debugging, different operating system boot requirements, product releases, etc.; rich device driver source code, such as serial port, Ethernet, SDRAM, FLASH, LCD, NVRAM, EEPROM, RTC, keyboard, etc.; relatively rich development debugging documentation and strong technical support; based on the above reasons, this article provides a detailed analysis of the current mainstream ARMv8 architecture U-Boot boot process to help everyone quickly learn and understand the U-Boot workflow.

2 ARMv8 U-Boot Boot Process

First, let’s look at a diagram provided by ARM: The above diagram summarizes the boot hierarchy structure recommended by ARM for ARMv8:

The official boot process is divided into stages: BL1, BL2, BL31, BL32, and BL33. In order, after the chip starts, it first executes the code of the BL1 stage, then verifies and starts BL2. BL2 starts BL31 or BL33 based on specific designs. BL32 only exists and is verified and loaded if BL31 is present.

ARMv8 is divided into Secure World and Non-Secure World (Normal World), with four exception levels from high to low being EL3, EL2, EL1, and EL0.

Secure World can execute trusted firmware and apps, such as password payments and fingerprint recognition services that rely on security guarantees. Non-Secure World is where common U-Boot, Linux, QNX, and other bare-metal programs or operating systems run.

EL3 has the highest management authority, responsible for security monitoring and switching security modes. EL2 mainly provides support for virtualization. EL1 is a privileged mode that can execute some privileged instructions for running various operating systems, while in secure mode, it runs a trusted OS. EL0 is the unprivileged mode, where all APP applications run.

The functions corresponding to BL1, BL2, BL31, BL32, and BL33 in the above diagram are as follows:

BL1: The root of all trust, generally a piece of boot loading code that is hard-coded in ROM, used to guide BL2 and verify its signature to ensure trusted execution; BL2: Generally a piece of trusted secure boot code in flash, its trust is established on the verification by BL1, mainly completing some platform-related initialization, such as DDR initialization, and after completing the initialization, looking for BL31 or BL33 to execute; if BL31 is found, BL33 will not be called; if BL31 is not found, BL33 must exist; BL31: Unlike BL1 and BL2, which are one-time runs, BL31 exists as the last trusted firmware, entering EL3 through the SMC instruction during system operation to call system security services or switch between Secure World and Non-Secure World; after completing BL31 initialization, it will look for BL32 or BL33 to verify and load for execution; BL32: OPTee OS + secure app, it is a trusted secure OS running in EL1 and starts trusted apps (such as fingerprint verification apps) in EL0, and after the Trust OS completes its operation, it returns to BL31 through the SMC instruction, and BL31 switches to Non-Secure World to continue executing BL33; BL33: Non-secure firmware, which is commonly UEFI firmware or U-Boot, and may also directly start the Linux kernel; the startup of BL1, BL2, BL31, and BL32 is a complete process of establishing the ATF trust chain (ARM Trusted Firmware), with common PSCI (Power State Coordination Interface) functions implemented in ATF’s BL31.

The last diagram fully displays the entire call process: BL2 can selectively load different firmware based on the existence of BL31 and BL32;

In summary, U-Boot is a bootloader that runs in the Non-Secure World, responsible for loading various operating systems and providing rich driver interfaces; and depending on the presence of secure firmware, it can have different boot processes, as shown below.

The relationship between U-Boot, U-Boot-SPL, and U-Boot-TPL: For general embedded systems, only one U-Boot as a bootloader is sufficient, but in cases of small memory or with ATF, there can also be SPL and TPL;

SPL: Secondary Program Loader, secondary loader; TPL: Tertiary Program Loader, tertiary loader. The emergence of SPL and TPL is primarily due to the system SRAM being too small, or ROM being unable to move all code from flash, eMMC, USB, etc., to SRAM for execution in one go when DDR is not initialized, or the flash being too small to fit the entire U-Boot for on-chip execution. Therefore, U-Boot defined SPL and TPL, which follow the same boot process as U-Boot, but in SPL and TPL, most drivers and functions are removed, keeping only a portion of the necessary functions controlled by CONFIG_SPL_BUILD and CONFIG_TPL_BUILD; generally, just using SPL is sufficient, as SPL completes DDR initialization and initializes some peripheral drivers, such as USB, eMMC, to load U-Boot from other peripheral devices. However, if SPL is still too large for small systems, TPL can be added, which only performs specific initializations like DDR to ensure a small code footprint, thus loading SPL from a specified location, and SPL then loads U-Boot.

Currently, SPL can replace BL2 in the above diagram or BL1, depending on the specific vendor’s implementation. Some chip vendors may hard-code SPL in ROM, giving it the ability to load U-Boot or other firmware from eMMC, USB, etc.

Of course, in the presence of ATF, U-Boot can be loaded by ATF, or SPL can load ATF, which then loads U-Boot. In fast-boot systems, SPL can directly load Linux or other operating systems, skipping U-Boot. In the above diagram, ARM only provides a suggested boot trust chain; the specific implementation needs to be determined by the chip vendor;

Subsequent analyses of the boot process will expand on the branching execution paths of SPL and TPL wherever they exist, aiming to analyze the functions of SPL and TPL as well;

3 U-Boot Source Code Structure and Some Compilation Configuration Methods

3.1 Compilation Configuration Methods

U-Boot uses the same compilation configuration method as Linux, namely using the Kbuild system to manage the overall code configuration and compilation, customizing various vendor chip bootloader binary programs through defconfig. When compiling, it is only necessary to introduce a cross-compilation tool through environment variables or command line parameters:

CROSS_COMPILE: Defines the cross-compilation toolchain, which can be aarch64-linux-gnu-, arm-none-eabi-, or ppc-linux-gnu-, etc.; U-Boot has several configurations that need to be defined by the corresponding board. SYS_ARCH, SYS_CPU, SYS_SOC, SYS_BOARD, SYS_VENDOR, SYS_CONFIG_NAME; generally, all can be defined in board/vendor/board/Kconfig, and some SYS_CPU and SYS_SOC can also be defined in arch/xxx/Kconfig. Based on these configurations, the CPU architecture, vendor, board-level information, and SoC information can be determined. The Makefile will automatically enter the corresponding directory based on the above information to organize the compilation rules. If there is no corresponding board information, it is necessary to create these Kconfig files and establish defconfig in the corresponding directory.

The configs directory contains all supported board configurations in U-Boot. For example, to use the configuration information for the rk3399 evb board, you can compile it as follows:

make CROSS_COMPILE=aarch64-linux-gnu- evb-rk3399_defconfig
make

If there is no corresponding defconfig, you can find a defconfig similar to your board-level information to generate a .config, then use menuconfig to complete your board configuration, and finally save it as your board’s defconfig through savedefconfig:

make CROSS_COMPILE=aarch64-linux-gnu- evb-rk3399_defconfig
make menuconfig
make savedefconfig
cp defconfig configs/my_defconfig

Below is the definition for evb rk3399:

CONFIG_SYS_ARCH="arm"
CONFIG_SYS_CPU="armv8"
CONFIG_SYS_SOC="rk3399"
CONFIG_SYS_VENDOR="rockchip"
CONFIG_SYS_BOARD="evb_rk3399"
CONFIG_SYS_CONFIG_NAME="evb_rk3399"

Based on the CONFIG_SYS_BOARD definition, a corresponding header file include/configs/xxxx.h will be automatically included for each source file. For evb rk3399, it will be include/configs/evb_rk3399.h. This header file can define some key configurations for the board, such as the system’s RAM size, the starting address and size of environment variables, GIC base address, clock frequency, whether to enable watchdog, etc., according to specific needs.

U-Boot uses Kconfig and include/configs/xxx.h to flexibly determine the U-Boot compilation process and the final generated files. For example, when CONFIG_SYS_CPU is defined as “armv8” and CONFIG_SYS_ARCH as “arm”, it determines the target architecture as armv8 and automatically enters the corresponding directory for compilation and linking according to the Makefile.

3.2 U-Boot Source Code Structure

Here, we will explain some commonly used directories:

arch: Contains the startup initialization process code for various architectures, linking scripts, etc.; board: Contains most of the vendor’s board initialization code, and basic platform-related code is located in the corresponding board directory. Early board code was in arch/xxx/xxx-mach, but now it is generally not placed under the arch directory; cmd: Contains the implementation of a large number of practical U-Boot commands, such as md, cp, cmp, tftp, fastboot, ext4load, etc., and we can also add our own implemented commands here; common: Contains the core initialization code for U-Boot, including board_f, board_r, SPL, and a series of codes; configs: Contains configuration files for all boards, which can be used directly; drivers: A large repository for driver codes; dts: Directory defining the compilation rules for generating dtb and embedding dtb into U-Boot; env: Implementation code for environment variables; fs: Implementation for reading and writing file systems, which includes implementations for various file systems; include: Path for storing all common header files; lib: Repository for implementing many general functions, provided for use by various modules; net: Implementation of network-related functionalities; scripts: Directory for storing scripts for compilation and configuration files; tools: Implementation of testing and utility tools, such as the implementation code for mkimage is located here;

4 U-Boot ARMv8 Linking Scripts

Before analyzing the source code, let’s first look at U-Boot’s linking script. By examining the linking script, we can understand the overall composition of a U-Boot and know what certain logic is accomplishing during the startup analysis. In ARMv8, U-Boot uses arch/arm/cpu/armv8/u-boot.lds for linking. U-Boot-SPL and U-Boot-TPL use arch/arm/cpu/armv8/u-boot-spl.lds for linking. Because each board’s situation may differ, U-Boot can customize u-boot-spl.lds and u-boot-tpl.lds through Kconfig.

4.1 u-boot.lds

/* SPDX-License-Identifier: GPL-2.0+ */
/*
 * (C) Copyright 2013
 * David Feng <[email protected]>
 *
 * (C) Copyright 2002
 * Gary Jennejohn, DENX Software Engineering, <[email protected]>
 */

#include <config.h>
#include <asm/psci.h>

OUTPUT_FORMAT("elf64-littleaarch64", "elf64-littleaarch64", "elf64-littleaarch64")
OUTPUT_ARCH(aarch64)
ENTRY(_start) -------------------------------------------------------------------- (1)
/*
 *（1）First, it defines the output format of the binary program as "elf64-littleaarch64", 
 *    the architecture is "aarch64", and the program entry is the symbol "_start";
 */
SECTIONS
{
#ifdef CONFIG_ARMV8_SECURE_BASE -------------------------------------------------- (2)
/*
 *（2）ARMV8_SECURE_BASE is U-Boot's support for PSCI. When defined, it can redirect the text, 
 *    data, and stack segments of PSCI to specific memory instead of embedding them in U-Boot.
 *    However, generally, vendor implementations will use ATF to separate it from the bootloader, so this function is not commonly used;
 */
 /DISCARD/ : { *(.rela._secure*) }
#endif
 . = 0x00000000; -------------------------------------------------------------- (3)
/*
 *（3）Defines the base address for program linking, which is defaulted to 0, and can be modified 
 *    through CONFIG_SYS_TEXT_BASE.
 */
 . = ALIGN(8);
 .text :
 {
  *(.__image_copy_start) --------------------------------------------------- (4)
/*
 *（4）__image_copy_start and __image_copy_end are used to define segments that need to be redirected, 
 *    U-Boot is a bootloader that is divided into pre-redirect initialization and post-redirect initialization, 
 *    so here it defines the starting and ending addresses of the data that need to be moved to DDR after completing the pre-redirect initialization;
 *
 *    Most of the time, U-Boot runs in limited SRAM or read-only flash, 
 *    U-Boot uniformly avoids accessing global variables before DDR initialization and relocation, 
 *    but in order to ensure that U-Boot can read and write global variables and memory, 
 *    and call various driver capabilities, U-Boot has divided the startup initialization into two parts: 
 *    pre-redirect initialization (board_f) and post-redirect initialization (board_r). During pre-redirect, 
 *    some necessary initializations are completed, including possible DDR initialization, 
 *    and then through __image_copy_start and __image_copy_end, U-Boot is moved to DDR, 
 *    and in DDR, it performs post-redirect initialization, at which point U-Boot can 
 *    normally access global variables and other information.
 * 
 *    If you want to read and write some global variable information during the board_f process, what should you do?
 *    U-Boot achieves this function by defining global_data (gd), and this will be detailed later when analyzed.
 */
  CPUDIR/start.o (.text*) -------------------------------------------------- (5)
/*
 *（5）Defines the header text segment of the linking program, ARMv8 is
 *    arch/arm/cpu/armv8/start.S,
 *    all text segments in start.S will be linked to this segment, and the segment entry symbol is _start;
 */
 }

 /* This needs to come before *(.text*) */
 .efi_runtime : { ------------------------------------------------------------ (6)
/*
 *（6）This segment will only appear when defining EFI runtime support, generally not of concern;
 */
        __efi_runtime_start = .;
  *(.text.efi_runtime*)
  *(.rodata.efi_runtime*)
  *(.data.efi_runtime*)
        __efi_runtime_stop = .;
 }

 .text_rest : ---------------------------------------------------------------- (7)
/*
 *（7）All other text segments except start.o will be linked to this segment;
 */
 {
  *(.text*)
 }

#ifdef CONFIG_ARMV8_PSCI -------------------------------------------------------- (8)
/*
 *（8）Similar to (2), is the support for PSCI-related functions, generally not used;
 */
 .__secure_start :
#ifndef CONFIG_ARMV8_SECURE_BASE
  ALIGN(CONSTANT(COMMONPAGESIZE))
#endif
 {
  KEEP(*(.__secure_start))
 }

#ifndef CONFIG_ARMV8_SECURE_BASE
#define CONFIG_ARMV8_SECURE_BASE
#define __ARMV8_PSCI_STACK_IN_RAM
#endif
 .secure_text CONFIG_ARMV8_SECURE_BASE :
  AT(ADDR(.__secure_start) + SIZEOF(.__secure_start))
 {
  *(._secure.text)
  . = ALIGN(8);
  __secure_svc_tbl_start = .;
  KEEP(*(._secure_svc_tbl_entries))
  __secure_svc_tbl_end = .;
 }

 .secure_data : AT(LOADADDR(.secure_text) + SIZEOF(.secure_text))
 {
  *(._secure.data)
 }

 .secure_stack ALIGN(ADDR(.secure_data) + SIZEOF(.secure_data),
       CONSTANT(COMMONPAGESIZE)) (NOLOAD) :
#ifdef __ARMV8_PSCI_STACK_IN_RAM
  AT(ADDR(.secure_stack))
#else
  AT(LOADADDR(.secure_data) + SIZEOF(.secure_data))
#endif
 {
  KEEP(*(.__secure_stack_start))

  . = . + CONFIG_ARMV8_PSCI_NR_CPUS * ARM_PSCI_STACK_SIZE;

  . = ALIGN(CONSTANT(COMMONPAGESIZE));

  KEEP(*(.__secure_stack_end))
 }
#endif

 . = ALIGN(8);
 .rodata : { *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*))) } ------------------- (9)
/*
 *（9）All read-only data will be aligned and stored in this segment;
 */

 . = ALIGN(8);
 .data : { -------------------------------------------------------------------- (10)
/*
 *（10）All data segments will be linked to this segment;
 */
  *(.data*)
 }

 . = ALIGN(8);

 . = .;

 . = ALIGN(8);
 .u_boot_list : { ------------------------------------------------------------- (11)
/*
 *（11）The u_boot_list segment defines all commands and device drivers currently supported in the system, 
 *     this segment collects commands and device drivers scattered across various files through a series of macro definitions 
 *     of U_BOOT_CMD and U_BOOT_DRIVER, and sorts them by name for quick retrieval and execution in the command line 
 *     and to detect registered devices and device tree matching to probe device driver initialization; 
 *     (Device driver probing is only effective when the DM modular driver is defined)
 */
  KEEP(*(SORT(.u_boot_list*)));
 }

 . = ALIGN(8);

 .efi_runtime_rel : {
                __efi_runtime_rel_start = .;
  *(.rel*.efi_runtime)
  *(.rel*.efi_runtime.*)
                __efi_runtime_rel_stop = .;
 }

 . = ALIGN(8);

 .image_copy_end :
 {
  *(.__image_copy_end)
 }

 . = ALIGN(8);

 .rel_dyn_start : -------------------------------------------------------- (12)
/*
 *（12）Generally, U-Boot runs based on the defined base address. If the loading address and linking address 
 *     are inconsistent, U-Boot cannot be executed. By configuring CONFIG_POSITION_INDEPENDENT, 
 *     this feature can be enabled, which will add the -PIE parameter when linking U-Boot. This parameter will generate rela* segments in the U-Boot ELF file. U-Boot will read the relative address values in this segment and traverse them to repair all addresses that need to be redirected, allowing it to run independently of address; 
 *     that is, regardless of how the linking base address is defined, U-Boot can run at any RAM address 
 *     (generally requiring a minimum of 4K or 64K address alignment);
 * 
 *     Note that this function can only be implemented on SRAM because it modifies the text and data segment addresses at runtime. 
 *     If it runs on on-chip flash at this time, it cannot write flash, causing the function to fail to achieve address independence;
 */
 {
  *(.__rel_dyn_start)
 }

 .rela.dyn : {
  *(.rela*)
 }

 .rel_dyn_end : {
  *(.__rel_dyn_end)
 }

 _end = .;

 . = ALIGN(8);

 .bss_start : { -------------------------------------------------------- (13)
/*
 *（13）The well-known bss segment;
 */
  KEEP(*(.__bss_start));
 }

 .bss : {
  *(.bss*)
   . = ALIGN(8);
 }

 .bss_end : {
  KEEP(*(.__bss_end));
 }

 /DISCARD/ : { *(.dynsym) } -------------------------------------------- (14)
/*
 *（14）Some segments that are useless during linking need to be discarded;
 */
 /DISCARD/ : { *(.dynstr*) }
 /DISCARD/ : { *(.dynamic*) }
 /DISCARD/ : { *(.plt*) }
 /DISCARD/ : { *(.interp*) }
 /DISCARD/ : { *(.gnu*) }

#ifdef CONFIG_LINUX_KERNEL_IMAGE_HEADER ----------------------------------- (15)
/*
 *（15）This is very useful when loading EFI, mainly adding some header information to U-Boot's binary header, 
 *     including size-endian, text segment size, etc., for EFI-related loaders to read information; 
 *     this header information does not belong to U-Boot but is simply appended;
 */
#include "linux-kernel-image-header-vars.h"
#endif
}

4.2 u-boot-spl.lds

This linking script is a standard SPL linking script and also includes the u_boot_list segment. If your board does not need command line or modular driver devices and is only used as a loader, a more simplified linking script can be customized.

/* SPDX-License-Identifier: GPL-2.0+ */
/*
 * (C) Copyright 2013
 * David Feng <[email protected]>
 *
 * (C) Copyright 2002
 * Gary Jennejohn, DENX Software Engineering, <[email protected]>
 *
 * (C) Copyright 2010
 * Texas Instruments, <www.ti.com>
 * Aneesh V <[email protected]>
 */

MEMORY { .sram : ORIGIN = IMAGE_TEXT_BASE, ---------------------------------------- (1)
/*
 *（1）>XXX format can place specified segments into the memory defined by XXX; 
 *    Generally, U-Boot-SPL has only a small runnable memory block, 
 *    so SPL will omit a lot of unnecessary segments, keeping only the key text and data segments, 
 *    and through >.sram, define the segments that are not used before DDR initialization into SDRAM, 
 *    and later only need to move these segments to DDR after completing DDR initialization, 
 *    without requiring additional address repair logic. For example, if there is an SRAM from 0x18000 to 0x19000, 
 *    and an SDRAM from 0x80000000 to 0x90000000, 
 *    then through >.sram, the map file might look like:
 *       0x18000 stext
 *       ...
 *       0x18100 sdata
 *       ...
 *       0x80000000 sbss
 *       ...
 */
  LENGTH = IMAGE_MAX_SIZE }
MEMORY { .sdram : ORIGIN = CONFIG_SPL_BSS_START_ADDR,
  LENGTH = CONFIG_SPL_BSS_MAX_SIZE }

OUTPUT_FORMAT("elf64-littleaarch64", "elf64-littleaarch64", "elf64-littleaarch64")
OUTPUT_ARCH(aarch64)
ENTRY(_start) -------------------------------------------------------------------- (2)
/*
 *（2）Same as u-boot.lds, sharing the same logic entry _start;
 */
SECTIONS
{
 .text : {
  . = ALIGN(8);
  *(.__image_copy_start) -------------------------------------------------- (3)
/*
 *（3）Similarly, if SPL needs redirection, this segment will be used to define it; 
 *    Most of the time, SPL will use redirection;
 */
  CPUDIR/start.o (.text*)
  *(.text*)
 } >.sram

 .rodata : {
  . = ALIGN(8);
  *(SORT_BY_ALIGNMENT(SORT_BY_NAME(.rodata*)))
 } >.sram

 .data : {
  . = ALIGN(8);
  *(.data*)
 } >.sram

#ifdef CONFIG_SPL_RECOVER_DATA_SECTION ---------------------------------------- (4)
/*
 *（4）The SPL_RECOVER_DATA_SECTION segment is used to save data segment data, 
 *    some boards modify the data segment data during initialization and restore the original data from this segment at a later stage;
 */
 .data_save : {
  *(.__data_save_start)
  . = SIZEOF(.data);
  *(.__data_save_end)
 } >.sram
#endif

 .u_boot_list : {
  . = ALIGN(8);
  KEEP(*(SORT(.u_boot_list*)));
 } >.sram

 .image_copy_end : {
  . = ALIGN(8);
  *(.__image_copy_end)
 } >.sram

 .end : {
  . = ALIGN(8);
  *(.__end)
 } >.sram

 _image_binary_end = .;

 .bss_start (NOLOAD) : {
  . = ALIGN(8);
  KEEP(*(.__bss_start));
 } >.sdram -------------------------------------------------------------- (5)
/*
 *（5）Defines the bss segment data into >.sdram, which can be directly cleared after initializing DDR 
 *    to use global uninitialized variables without side effects.
 */

 .bss (NOLOAD) : {
  *(.bss*)
   . = ALIGN(8);
 } >.sdram

 .bss_end (NOLOAD) : {
  KEEP(*(.__bss_end));
 } >.sdram

 /DISCARD/ : { *(.rela*) }
 /DISCARD/ : { *(.dynsym) }
 /DISCARD/ : { *(.dynstr*) }
 /DISCARD/ : { *(.dynamic*) }
 /DISCARD/ : { *(.plt*) }
 /DISCARD/ : { *(.interp*) }
 /DISCARD/ : { *(.gnu*) }
}

From the above linking scripts, it can be seen that the boot of ARMv8 U-Boot starts from arch/arm/cpu/armv8/start.S with _start and subsequently calls many address symbol tables defined in the linking scripts.

Original link: https://blog.csdn.net/maybeYoc/article/details/122937844