Core Components of OpenWrt: libubox (2): Utils

The utils module in libubox is difficult to categorize because its functions resemble a mixed bag, so I refer to it as miscellaneous tools. It provides some simple utility functions, such as endianness conversion, bit field operations, compiler attribute wrappers, sequential memory allocation functions (calloc_a), static array size macros, assertion/error utility functions, clock time retrieval wrapped for Apple compatibility, and base64 encoding/decoding.This article mainly provides a brief introduction to some utility functions in utils, and most importantly, it offers a simple code framework for using the libubox library, allowing everyone to practice and test various libubox library function interfaces.As the libubox version upgrades, the supported library function interfaces increase. To ensure universality, the subsequent discussions are based on OpenWrt 17.01 (LEDE) version, and the version I am using is libubox-2018-01-07 for your reference.The utils module provides many function interfaces. We will start analyzing from the utils.h file, where we can view its exported macro definitions, functions, and data structures.The calloc_a() function is used to allocate a block of memory large enough to hold multiple aligned objects. The return value of this function is a pointer to the starting position of the entire block of memory, while pointers to other objects are stored in the address parameter list. Below is the declaration of this function in utils.h:

/* * calloc_a(size_t len, [void **addr, size_t len,...], NULL) * * allocate a block of memory big enough to hold multiple aligned objects. * the pointer to the full object (starting with the first chunk) is returned, * all other pointers are stored in the locations behind extra addr arguments. * the last argument needs to be a NULL pointer */
#define calloc_a(len, ...) __calloc_a(len, ##__VA_ARGS__, NULL)
void *__calloc_a(size_t len, ...);

So how is this function used? We can look at a real case:netifd-2017-01-25-650758b1/wireless.c +520

static void wireless_add_handler(const char *script, const char *name, json_object *obj){
    struct wireless_driver *drv;
    char *name_str, *script_str;
    json_object *dev_config_obj, *iface_config_obj;
    struct uci_blob_param_list *dev_config, *iface_config;
    dev_config_obj = json_get_field(obj, "device", json_type_array);
    iface_config_obj = json_get_field(obj, "iface", json_type_array);
    if (!dev_config_obj || !iface_config_obj)
        return;
    drv = calloc_a(sizeof(*drv),
        &dev_config, sizeof(*dev_config) + sizeof(void *),
        &iface_config, sizeof(*iface_config) + sizeof(void *),
        &name_str, strlen(name) + 1,
        &script_str, strlen(script) + 1);
    drv->name = strcpy(name_str, name);
    drv->script = strcpy(script_str, script);
    dev_config->n_next = 1;
    dev_config->next[0] = &wdev_param;
    drv->device.config = dev_config;
    iface_config->n_next = 1;
    iface_config->next[0] = &vif_param;
    drv->interface.config = iface_config;
    ...

It can be seen that multiple structures and string types of memory are continuously allocated here, and the pointer to the first object can be directly used as the return value, while other objects require passing a double pointer along with the size of the object (an extra pointer size is allocated). This method allows for convenient simultaneous memory allocation for multiple objects, making the code more compact. However, there is no error handling; how is it ensured that the allocation is successful?It is important to note that although memory can be allocated for multiple variables using calloc_a, the memory cannot be released all at once; each variable must be freed individually using the free() function.The ARRAY_SIZE macro is used to calculate the size of an array, defined as follows:

#ifndef ARRAY_SIZE
#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
#endif

It is necessary to ensure that the object being calculated is an explicit array; if it is a pointer, the actual size cannot be accurately calculated.The clock_gettime function is implemented here to retrieve time, adapted for different platforms. Its declaration in utils.h is as follows:

#if defined(__APPLE__) && !defined(CLOCK_MONOTONIC)
#define LIBUBOX_COMPAT_CLOCK_GETTIME
#include <mach/clock_types.h>
#define CLOCK_REALTIME CALENDAR_CLOCK
#define CLOCK_MONOTONIC SYSTEM_CLOCK
int clock_gettime(int type, struct timespec *tv);
#endif

Byte order conversion macros are defined in the utils.h header file, which can be used directly:

#define const_cpu_to_be64(x) __constant_swap64(x)
#define const_cpu_to_be32(x) __constant_swap32(x)
#define const_cpu_to_be16(x) __constant_swap16(x)
#define const_be64_to_cpu(x) __constant_swap64(x)
#define const_be32_to_cpu(x) __constant_swap32(x)
#define const_be16_to_cpu(x) __constant_swap16(x)
#define const_cpu_to_le64(x) (x)
#define const_cpu_to_le32(x) (x)
#define const_cpu_to_le16(x) (x)
#define const_le64_to_cpu(x) (x)
#define const_le32_to_cpu(x) (x)
#define const_le16_to_cpu(x) (x)...

Compiler attribute modifiers are also provided in the utils.h header file, mainly utilizing GNU C __attribute__ to set various attributes:

#ifndef __packed
#define __packed __attribute__((packed))
#endif
#ifndef __constructor
#define __constructor __attribute__((constructor))
#endif
#ifndef __destructor
#define __destructor __attribute__((destructor))
#endif
#ifndef __hidden
#define __hidden __attribute__((visibility("hidden")))
#endif

Bit manipulation functions are provided here, defined as follows:

#ifndef BITS_PER_LONG
#define BITS_PER_LONG (8 * sizeof(unsigned long))
#endif
#define BITFIELD_SIZE(_n) (((_n) + (BITS_PER_LONG - 1)) / BITS_PER_LONG)
static inline void bitfield_set(unsigned long *bits, int bit){
	bits[bit / BITS_PER_LONG] |= (1UL << (bit % BITS_PER_LONG));}
static inline bool bitfield_test(unsigned long *bits, int bit){return !!(bits[bit / BITS_PER_LONG] && (1UL << (bit % BITS_PER_LONG)));}

The bitfield_set function is powerful in that it can set more than one unsigned long can hold, although it is rarely used.The BASE64 encoding and decoding capability is provided. If you are not familiar with what BASE64 encoding is, you can look it up online. In simple terms, it is a method of converting binary content into a string of 64 visible characters. The function interfaces are as follows:

int b64_encode(const void *src, size_t src_len, void *dest, size_t dest_len);
int b64_decode(const void *src, void *dest, size_t dest_len);
#define B64_ENCODE_LEN(_len) (((( _len) + 2) / 3) * 4 + 1)
#define B64_DECODE_LEN(_len) (((_len) / 4) * 3 + 1)

The cbuf functionality has not been seen used in other applications, with the provided interfaces as follows:

static inline unsigned int cbuf_order(unsigned int x){return 32 - __builtin_clz(x - 1);}
static inline unsigned long cbuf_size(int order){unsigned long page_size = sysconf(_SC_PAGESIZE);
unsigned long ret = 1ULL << order;
if (ret < page_size) ret = page_size;
return ret;}
void *cbuf_alloc(unsigned int order);
void cbuf_free(void *ptr, unsigned int order);

Example ProgramNext, I will demonstrate how to use the libubox library in a real project. To facilitate learning, I plan to rewrite the previously introduced HelloWorld program. If you are not familiar with it, please refer to the following:Write a HelloWorld ProgramNow, let’s get started. First, create a software package directory structure named demo-libubox:

ruok@ruok-vm:~/Downloads/openwrt/package/utils$ tree -L 2 demo-libubox
demo-libubox/
├── Makefile
└── src
    ├── main.c
    └── Makefile
1 directory, 3 files

The top-level Makefile describes this package, with the following content:

## Copyright (C) 2025
## This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.

include $(TOPDIR)/rules.mk
PKG_NAME:=demo-libubox
PKG_RELEASE:=1

include $(INCLUDE_DIR)/package.mk
TARGET_CFLAGS += -I$(STAGING_DIR)/usr/include -L$(STAGING_DIR)/usr/lib -lubox -lubus

define Package/demo-libubox
  SECTION:=utils
  CATEGORY:=Base system
  TITLE:=A libubox demo program
  MAINTAINER:=somebody
  DEPENDS:=+libubox +libblobmsg-json +libubus
endef

define Package/demo-libubox/description
  This package contains a utility that allows testing the libubox library.
endef

define Build/Compile
        $(MAKE) -C $(PKG_BUILD_DIR) \
                CC="$(TARGET_CC)" \
                CFLAGS="$(TARGET_CFLAGS) -Wall"
endef

define Package/demo-libubox/install
        $(INSTALL_DIR) $(1)/usr/bin
        $(INSTALL_BIN) $(PKG_BUILD_DIR)/demo-libubox $(1)/usr/bin
endef

$(eval $(call BuildPackage,demo-libubox))

The differences from HelloWorld may not be very intuitive. Let’s compare them using Beyond Compare:Core Components of OpenWrt: libubox (2): UtilsThere are mainly two differences: first, the TARGET_CFLAGS variable needs to include the header file reference path and link the libubox library; second, the software package dependencies need to include libubox, ensuring that the dependent libraries are compiled beforehand when compiling this package.Next is the source program main.c, which can be written according to your needs. This chapter mainly introduces the utils module, so I will provide a base64 encoding and decoding example program:

#include <stdint.h> // uint32_t
#include <stdio.h>
#include <stdlib.h> // free()
#include <string.h>
#include <libubox/utils.h>

int main(int argc, char **argv) {
    char buf[256], dest[256];
    uint8_t data[] = {0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A};
    printf("encode length: %d\n", B64_ENCODE_LEN(ARRAY_SIZE(data)));
    if (b64_encode(data, ARRAY_SIZE(data), buf, 256) <= 0) {
        printf("encode error!\n");
        return -1;
    }
    printf("encode result: %s\n", buf);
    printf("decode length: %d\n", B64_DECODE_LEN(strlen(buf)));
    if (b64_decode(buf, dest, 256) <= 0) {
        printf("decode error!\n");
        return -1;
    }
    printf("decode result:\n");
    for (int i = 0; i < 256; i++) {
        if (dest[i] == 0) break;
        printf("%X ", dest[i]);
    }
    return 0;
}

Finally, the content of the src/Makefile needs to be written, which controls the actual compilation actions of the source code. The content is also very simple:

all: demo-libubox
demo-libubox:        $(CC) $(CFLAGS) -Wall main.c -o $@ $^
clean:        rm -f demo-libubox *.o

How do we compile this program? We need to place the software package in the package directory or its subdirectory, for example, I placed it in the package/utils directory (you can adjust it according to your preference), and then execute make menuconfig to select the software package; otherwise, the compilation process will not generate the ipk upgrade package.Core Components of OpenWrt: libubox (2): UtilsFinally, execute the entire firmware compilation or compile the software package separately:

make package/utils/demo-libubox/{clean,prepare,compile} V=s -j1

The output of running the above program on the device is as follows:

root@LEDE:/tmp# demo-libubox
encode length: 17
encode result: AQIDBAUGBwgJCg==
decode length: 13
decode result: 1 2 3 4 5 6 7 8 9 A 

Thus, the content regarding the utils module in libubox has been introduced. For the software package construction part, I recommend interested students to try it out themselves. In the future, I will write example programs based on this software package regarding libubox.Series of Articles on Core Components of OpenWrt: Core Components of OpenWrt: libubox (1): Project Introduction

Leave a Comment