Memory Management Strategies for Embedded Systems

In embedded systems, memory is a scarce resource, especially in consumer products. To save costs, hardware resources are generally utilized to the fullest. During the development process, we often encounter situations where the running memory (RAM) is almost sufficient but just not enough, such as:

Need to add a small algorithm to the original system
OTA can only place firmware in memory
Need to dynamically allocate a relatively large space
Need to place some relatively large temporary files

If your device’s memory is already used up to 90% or even higher, achieving the above functions will likely affect the overall performance of the system and may even cause system anomalies. So, what can be done?

Without considering hardware upgrades to increase the RAM size, are there any other software methods to squeeze out a little space? The answer is yes, but it requires a comprehensive assessment of the problems and risks involved.

Here, we take the Junzheng T31ZC as an example to introduce its memory usage and how to squeeze out more memory space.

1. Memory Usage Analysis

T31ZC is a main processor based on the MIPS architecture, integrated with 512Mbit of DDR2 memory, and uses the Linux operating system.

1. Physical Memory Distribution

512Mbit of physical memory, which is 64MByte, is divided into two parts during actual use: rmem and mem

rmem is used for multimedia, such as audio and video encoding, cropping, OSD, etc.
mem is used for the Linux system memory

It is set through the tag part of the cmdline:

[root@Zeratul:~]# cat /proc/cmdline 
console=ttyS0,115200n8 mem=40M@0x0 rmem=24M@0x2800000 root=/dev/ram0 rw rdinit=/linuxrc mtdparts=jz_sfc:256K(boot),352K(tag),5M(kernel),6M(rootfs),2560K(recovery),1440K(system),512K(config),16M@0(all) lpj=6955008 quiet senv;[HW];init_vw=1920;init_vh=1080;nrvbs=2;mode=0;eenv; lzo_size=5907415 rd_start=0x80600000 rd_size=0xd35c00
[root@Zeratul:~]#

Here we see that 40M is allocated to the Linux system, and 24M is allocated for multimedia.

By using the dmesg command, we can see more about memory usage:

Memory Management Strategies for Embedded Systems

rd_start=0x80600000 rd_size=0xd35c00

This is used to store the starting position of the root filesystem and the size of that space.

Memory: 20680k/40960k

40960K is the total size available to the Linux system, which is the 40MB, and 20680k is the memory that the system can actually use.

The remaining 40960K - 20680k = 20280K memory, where did it go?

3868k kernel code, 20280k reserved, 1052k data, 196k init, 0k highmem

Part of it is used by the kernel, including the kernel code segment, data segment, and init segment.

The remaining 20280k - 3868K -1052K - 196K = 15164K

is reserved for ramfs usage.

2. mem Usage

Linux system starts, without running other applications, we check the memory usage as follows:

cat /proc/meminfo

The actual physical memory size available is still 12872 kB

Memory allocated for the Linux system is 40M, but before the applications start running, there is only 12872K left. Where did the memory go!

Summary is as follows:

Total physical memory is 64M, 24M allocated for multimedia, and 40M allocated for the Linux system.

Total Size	Multimedia Memory	Linux System
64M	24M	40M

Linux system memory can use is 20680k, and the reserved memory is 20280k

Linux System	Available	Reserved
40960k	20680k	20280k

Linux system available memory is occupied by kernel modules, buffer cache, and other system services, which takes up 7768K, leaving the rest for application use.

Available	MemFree	Modules/Others
20680k	12912k	7768K

Linux system reserved memory is mainly reserved for the kernel and the root filesystem, where the kernel mainly includes the code segment, data segment, and init segment. The remaining is mainly the space occupied by the root filesystem.

Reserved	Kernel Code	Kernel Data	Kernel Init	Ramfs	Others
20280k	3868K	1052K	196K	13527K	1637K

2. Optimization Directions

From the above analysis, the directions for optimization are:

Reduce module loading and system services
Kernel optimization
Rootfs optimization

1. Reduce Module Loading and System Services

This space for optimization is limited, and will not be discussed in detail here. You can check the actual module loading situation with the command: cat /proc/modules

2. Kernel Optimization

Removing the kernel’s debugging information can save a small amount of space by unchecking the Load all symbols for debugging/ksymoops option in menuconfig.

3. Rootfs Optimization

From the above analysis, we see that the space reserved for ramfs is 13527K, which is a very large space. From the startup cmdline, we see the partition information of Flash as follows:

256K(boot),352K(tag),5M(kernel),6M(rootfs),2560K(recovery),1440K(system),512K(config),16M@0(all)

(1) The space reserved for rootfs in Flash is 6M, while the memory reserved for rootfs is 13527K. Why is there such a large discrepancy?

The main reason is that our rootfs is compressed when burned to Flash, and when loading the rootfs, it is first read from Flash and then decompressed to a specified address in memory before mounting the decompressed rootfs as a ramfs filesystem.

rootfs packaging command is as follows:

cd ./_rootfs_camera
find . | cpio -H newc -o &gt; ../rootfs_camera.cpio
cd ..
lzop -9 -f rootfs_camera.cpio -o rootfs_camera.cpio.lzo
./mark_rootfs_pc rootfs_camera.cpio.lzo

Archive all files in the _rootfs_camera directory to the rootfs_camera.cpio file in the previous directory
Use the lzop command with the highest level (9) compression to compress the rootfs_camera.cpio to the rootfs_camera.cpio.lzo file, and -f indicates forced execution, which means overwriting it if it already exists.
mark_rootfs_pc is used to update the actual file size of rootfs_camera.cpio.lzo, which actually writes the compressed file size to the beginning of rootfs_camera.cpio.lzo (4 bytes).

(2) Why use the ramfs filesystem?

ramfs is suitable for temporary storage, temporary files, kernel module loading, and other temporary applications where data does not need to be permanently saved. It is a virtual filesystem in the Linux kernel and does not require specific mount options.

The advantages are:

Fast read and write operations
Zero latency
Lightweight
Easy to create and destroy

The disadvantages are:

No persistence
Memory limitations
Not suitable for large files
Requires sufficient memory

The Junzheng T31ZC is a low-power SOC that requires high real-time performance and fast startup. The typical application scenario is to have another MCU control its power on and off, and when an event is triggered, the SOC powers on to handle the event. After the event is processed, the SOC powers off, leaving only the MCU to work, achieving the goal of power saving, so using ramfs is also a reasonable choice.

The official manual suggests:

Only store necessary programs and library files that require fast startup in rootfs. For programs or library files that do not require fast startup, it is recommended to place them in the system partition to achieve quick startup and save memory.

3. Alternative Solutions

1. Problem Analysis

According to the above analysis, if the system uses ramfs, the largest optimization space is the root filesystem. Reducing the size of the root filesystem can directly save memory.

In the official suggestions above, there is another problem to face, which is that after the application starts, it needs to use the dl_open to load the required dynamic libraries from the system partition. If your program primarily uses static libraries, this method will not achieve the desired effect.

Check the contents of the rootfs filesystem, the directories that occupy a large space are stone and lib.

./stone directory contains the executable main program
./lib directory mainly contains some dynamic libraries, ko driver files, and a wifi used bin file

-rwxrwxr-x  1 biao biao 6.2M Oct 15 18:14 main
-rwxrwxrwx 1 biao biao 1003K Aug 21 10:04 cywdhd.ko
-rwxrwxrwx 1 biao biao 404K  Aug 21 10:04 fw_bcm43438a1.bin

2. Solution

main executable program executes upon startup:

./etc/init.d/rcS:/stone/main &

wifi driver is also loaded at startup:

insmod /lib/modules/cywdhd.ko firmware_path=/lib/firmware/fw_bcm43438a1.bin nvram_path=/lib/firmware/nvram.txt iface_name=wlan0

Can it be done this way:

After the main executable, the cywdhd.ko driver, and the firmware fw_bcm43438a1.bin are loaded at startup, can they be deleted to free up memory?

Since these files are placed in memory, deleting them will not affect the files in Flash, and they can be read from Flash again when powered on next time, so deleting them can indeed free up some memory.

In my example above, deleting the main file can free up 6.2M of space, which can indeed achieve the goal of freeing memory.

However, does this operation carry risks?

4. Does Deleting ELF Files Have an Impact?

We know that programs are loaded into memory using segmented paging, so how can we know when deleting the main executable file that all contents of the main program have been loaded into memory?

Similarly, deleting the ko files and bin firmware data files will encounter the same question.

1. Deleting cywdhd.ko Driver

By using the cat /proc/modules command, we can view the loading status of the module:

[root@Zeratul:bin]# cat /proc/modules | grep cywdhd
cywdhd 671712 0 - Live 0xc0215000
jzmmc 17113 1 cywdhd, Live 0xc010f000
mmc_core 90139 3 mmc_block,cywdhd,jzmmc, Live 0xc00e5000

We see that cywdhd is loaded into memory at the address 0xc0215000, and Live indicates that it has been loaded into the kernel and is running. 671712 indicates its size.

Why is it that we previously saw that cywdhd.ko had a size of 1003K, but when loaded into the kernel, it is only 67171?

We use the file command to check the information of the cywdhd.ko file, and find that it is in ELF file format and is not stripped, meaning it still contains some debugging information and debugging symbol tables.

biao@ubuntu: file cywdhd.ko 
cywdhd.ko: ELF 32-bit LSB relocatable, MIPS, MIPS32 version 1 (SYSV), BuildID[sha1]=a00e1928dd77ac04bb813b22cd485156f3392741, not stripped

strip processing reduces the file size to 657788

biao@ubuntu: mips-linux-uclibc-gnu-strip cywdhd.ko 
biao@ubuntu:ll
-rwxrwxrwx 1 biao biao 657788 Oct 15 19:42 cywdhd.ko

Based on the information above, we can roughly determine that deleting the cywdhd.ko file carries no risk.

Another way to confirm is to check the information in the /proc/kallsyms file.

/proc/kallsyms contains kernel symbol table information, including the address range of loaded modules. If the address range of the module includes the entire module, then it is usually fully loaded into memory.

2. Deleting fw_bcm43438a1.bin Firmware

fw_bcm43438a1.bin is a firmware loaded onto the kernel as a driver for cywdhd.ko, and when it is used or whether it is fully loaded into memory depends on the cywdhd.ko driver.

Therefore, I personally believe that deleting fw_bcm43438a1.bin carries a significant risk.

3. Deleting the main Program

In Linux systems, executable files, dynamic libraries, and driver files are all in ELF file format.ELF files are divided into many segments, commonly including code segments, data segments, and BSS segments.

We can use the command readelf -S your_program to view the details:

We see that the size of the code segment is 0x46a6b0. In many operating systems, to save memory space, they use dynamic loading, i.e., segmented paging loading.

Based on the principle of locality, programs are generally loaded into memory with the data close to the current execution position.

However, for the ramfs filesystem, since it has already been fully loaded into memory, when the main file runs, is it loaded all at once (global loading) or on demand (segmented paging loading)?

If it is loaded all at once during execution, then deleting the main file after it runs is safe; otherwise, it will introduce system risks.

As for the loading of executable files on the ramfs filesystem, I have not found more detailed reference materials. Colleagues familiar with this field can provide some suggestions.

5. Manually Releasing Memory

Memory can be manually released using the command:

echo 1 > /proc/sys/vm/drop_caches

Information about drop_caches can be understood through the introduction in man proc, and will not be elaborated here:

6. Conclusion

In summary, in the case of insufficient memory in embedded Linux systems, the following methods can be used for optimization:

Remove debugging information symbol tables from the kernel to reduce the size of the kernel image file
Use strip processing on ELF files on the filesystem (including executable files, dynamic libraries, and ko driver files) to remove unnecessary information
Place dynamic libraries in other partitions and load them using dl_open after the program runs
After using executable files or driver files, delete them (this may carry risks)
Manually release memory

Which method to use can be evaluated and selected based on actual usage conditions.

If there are any errors in the analysis above, please feel free to provide criticism and corrections.

END

Source: liwen01

Copyright belongs to the original author. If there is any infringement, please contact for deletion.

▍Recommended Reading

Getting scolded for “two floating point numbers being equal”

Kirin 9000s, not from SMIC, but…

The words programmers are most likely to misread, I exploded when I heard status

→ Follow for updates ←