Why Major Companies Use Yocto?

Hello everyone, I am Fish Eagle.

1. The Bitter History of Buildroot

Before this, my boss had just asked me to learn about Buildroot to compile kernel images because we had an old project that used it for image compilation.

In the past, these images were maintained by foreigners (I am currently working in a Fortune 500 foreign company), and we only needed to retrieve the images from a specified address and use tools to burn the SD card images.

Later, my boss wanted the images to be maintained domestically and to be able to compile them ourselves, so he asked me to learn image compilation (I had previously discussed my career direction with my boss, and he knew I wanted to develop in the Linux direction, so he gave me this opportunity).

I had learned about Buildroot many years ago and had compiled it, thinking it wouldn’t be particularly difficult and that I would complete the task quickly.

However, I didn’t expect it to take more than half a month to get it done. The most impressive part was the download of around 2.5 GB of open-source resources. In the past, I had never encountered these issues while compiling images; it usually took half a day to a day.

Looking back now, there are two reasons for this:

1. Back then, GitHub was accessible, and many foreign resources could be downloaded easily.

2. The current images depend on too many open-source libraries, many of which are old. Previously, I was only learning the Linux kernel and only needed the basic components.

Thus, compiling uboot, kernel, and file system (the three major components) went smoothly.

Now, some resources are just too slow to download, and I could only look for resources in various ways. Moreover, transferring from Gitee, I finally managed to complete the downloads (thankfully, it supports offline installation).

However, I still couldn’t install the Qt functionality in the end. I don’t know why; since it wasn’t important, I didn’t bother with it.

Moreover, many resources have specified hash values, and you can only download resources with the same hash value. Otherwise, even if you modify the hash value to adapt to new resources, it may ultimately lead to system incompatibility.

2. The Difficulties of Yocto

IMX_YOCTO_PROJECT_USERS_GUIDE.pdf

In the end, when generating the SDK, I got stuck at the last step, unable to find the dnf Python tool. Thankfully, I generated the toolchain through other commands; otherwise, if I got stuck at the last step of compiling the image, it would have been a big problem (later, Docker solved these pain points).

bitbake meta-toolchain

This left me with no temper; I could only seek help from the supplier and got the resources he had downloaded in advance.

With the resources complete, I could finally start compiling.

The compilation process went relatively smoothly, but compared to Buildroot, it was extremely slow, and resource consumption was enormous (a very simple process might require hundreds of MB of resources, which explains why the entire image needs over 100 GB of space).

Here’s a very useful command that allows other tasks to continue running when an error occurs without immediately interrupting:

bitbake xxx -k

Additionally, I recommend a Yocto introductory video: Bilibili: iMX6ULL Builds Embedded Linux System Using Yocto

Flexibility and Customizability: The Yocto Project provides a flexible toolset and development environment, allowing developers to freely choose resource usage or improve specific components (including specific versions of these components). This flexibility enables developers to customize Linux distributions according to their needs, thus meeting the requirements of specific hardware platforms. Avoiding Redundant Work: The Yocto Project encourages innovation across various applications and device types, supporting multiple hardware architectures such as x86, ARM, RISC-V, PPC, and MIPS. By sharing technologies, software stacks, configurations, and best practices, the Yocto Project helps developers avoid starting from scratch, reducing redundant work and unnecessary maintenance. Powerful Build System: The core of the Yocto Project is BitBake, an efficient task scheduler and execution system that can handle complex dependencies and optimize the build process. BitBake ensures that build tasks are executed in the correct order, thus improving build efficiency and predictability. Rich Ecosystem: The Yocto Project has an active community and abundant documentation, providing strong support for developers. Community members include hardware manufacturers, operating system vendors, and independent consultants, all contributing resources and information to form a robust ecosystem. Standardization and Compatibility: The Yocto Project provides standardized build processes and tools, facilitating collaboration between different developers and teams. By using the Yocto Project, developers can ensure that their customized Linux systems have high compatibility and portability across different hardware platforms. Optimized Resource Utilization: The Yocto Project allows developers to precisely create the content required for embedded devices, only adding the necessary functional support or packages. This helps optimize system performance, reduces unnecessary resource occupancy, and provides more efficient solutions for embedded and IoT devices. Maintainability and Scalability: The design of the Yocto Project allows users to use self-developed operating systems or commercial operating systems without losing optimizations or functionalities during migration. Additionally, commercial operating systems can also use the Yocto Project as upstream resources, ensuring maximum code reuse. Supporting Multiple Build Targets: The Yocto Project can not only be used to create Linux distributions but can also generate real-time operating system (RTOS) toolchains for bare-metal development, as demonstrated by the Zephyr project.

Next, let’s discuss why the Yocto method is good based on actual development experience.

1. Development Environment

Docker technology has solved the problem of environment setup, so environment setup is no longer an issue.

When I was young, I didn’t know how good Docker was.

2. Can Compile Various Codes

We know that a project involves various programming languages, such as C/C++, Python, Java, etc., and various platforms like u-boot, Linux, MCU, qt, across different architectures like ARM, RISC-V, x86, etc. Yocto can compile all these codes and package them together for easy access.

Moreover, the same set of meta can easily compile for both x86 and ARM, allowing for development and debugging work even without an ARM board. Automated testing also becomes easier.

3. Supports QEMU Debugging

Native support for QEMU debugging; however, I didn’t quite understand this part. Once I do, it will significantly improve development efficiency.

4. Customization

This is probably the reason why major companies like Yocto.

Often, our work is built on the foundation of others. For example, if I want to add a Qt interface in Ubuntu. At this time, if there is an existing Ubuntu, adding Qt becomes much easier.

Similarly, the Yocto project contains many Linux distributions with various components (possibly hundreds or thousands), allowing for the addition or deletion of certain components at will, and the versions of these components are maintained and updated by the community.

For instance, a specific version of a tool like vim may require a certain version of the Linux kernel. The compatibility of these commonly used tools is handled, and you only need to download a poky (the reference implementation of Yocto) distribution, specify a version, and you can compile the entire image (though downloading may take some time since the versions of the dependent components may differ, requiring redownload). Chip manufacturers will also add their own special customizations based on these foundations.

In addition to the overall image customization, individual components and single codes can also be modified. For instance, modifying device trees, kernel codes, and drivers can all be added to the base Linux code as patches (without modifying the source code).

Moreover, upgrading the Linux kernel to the latest version is also very easy. Generally, Poky will synchronize the latest kernel and upgrade the dependent components together, so your project modifications might just be a change in a commit of Poky (if kernel code is modified, it may require some synchronization changes), but those components, such as vim, nano, ssh, etc., will be upgraded without needing your concern.

Yocto-generated images also have various options, such as:

core-image-minimal: This is a very basic image containing necessary system tools and the Linux kernel, suitable as a base for customizing more complex systems. core-image-sato: This is a standard image optimized for desktop environments, containing more software packages and tools, suitable for desktop or display devices. core-image-full-cmdline: This image contains a complete command line toolset but does not include a graphical interface, suitable for embedded devices requiring a full command line environment. core-image-multimedia: This image contains multimedia-related software packages, suitable for devices requiring audio and video processing capabilities.

And if there are some component codes you don’t like, you can modify them anytime; these modifications won’t change the source code but will be applied as patches to another compilation environment, ensuring the purity of the source code.

5. Practicality of the Project

If you have a Linux board V1 and have adapted the code based on the current hardware, and then discover some issues requiring hardware changes, turning it into V2, while possibly also needing to synchronize modifications with other MCU codes, and the old version board is still in use.

How to handle these dependency issues?

The old method is to manually maintain the differences in environment dependencies, or a more advanced method is to use scripts or repo tools for maintenance.

Yocto, with the help of the kas tool, can perfectly handle this situation. The meta repository can have information for both board types on the same branch; you just need to set up the required machine type before compiling to handle the corresponding dependencies, ensuring no incompatibility between Linux and MCU occurs anymore.

Moreover, using kas allows for differential modifications on V2 based on V1, only needing to add the differences, avoiding redundant work or copying or branching.

6. SDK

It is easy to generate various SDKs, which, after installation, are suitable for upper-level application development without occupying too much space, so you won’t have to install the entire Yocto development environment for a small feature.

Standard SDK: This SDK includes artifacts for application development, whether bootloaders, Linux kernel development, or other user-space software. It provides a cross-development toolchain and libraries optimized for specific images. You can use the standard SDK to independently develop and test code intended to run on a target machine. Extensible SDK: In addition to all the features of the standard SDK, the extensible SDK provides tools that allow you to easily add new applications and libraries to the image, modify the source code of existing components, test changes on the target hardware, and easily integrate applications into the OpenEmbedded build system. The extensible SDK can be installed on any machine and can be used to develop applications, images, and kernels. Native SDK: This SDK targets running on the target device, suitable for cases where the embedded device is powerful enough to be used as a development environment. The native SDK includes headers, libraries, and toolchains, allowing direct application development and testing on the target device. Cross-development SDK: This SDK is used to generate binaries for the target machine on the development host. It includes cross-compilation toolchains, libraries, headers, binaries, and possibly other utilities and applications.

7. Ecosystem

There are many open-source meta layers available for use, and many chip manufacturers have adapted to the Yocto environment, supported by numerous developers maintaining Yocto, resulting in a robust ecosystem.

Finally, let’s discuss the drawbacks:

1. High resource consumption (each task has a separate compilation environment);

2. The speed is very slow when starting compilation because it needs to compile various tools on the host;

3. Difficult to get started, steep learning curve, but suitable for maintaining complex projects;

4. Resource nesting is deep, making it difficult to find and understand the compilation process.

END