Click on the “Embedded Application Research Institute” above, select “Pin/Star the Official Account“

Practical Benefits, Delivered First-Hand!

Source | CSDN Passing Bear

Compiled & Formatted | Embedded Application Research Institute

1. Introduction

Those involved in embedded product development must have encountered the need to save device parameters. A common approach is to manage parameters using structures and save the entire structure data in Flash for easy retrieval later.

1.1 Purpose

This article analyzes the advantages and disadvantages of several methods for parameter storage in embedded/microcontroller systems (specifically for microcontroller/embedded development) and addresses some shortcomings of the structure-based approach (focused on Section 3).

2. Parameter Storage Formats

2.1 Structure Format

This method is the most commonly used in embedded/microcontroller development, defining all system parameters using structures, and then saving data. Here are the advantages and disadvantages of this method.

Storage Method: Binary bin file format

Advantages:

Simple Management: Parameters can be managed easily without additional code. Minimal Memory Usage: Stored in Flash using structures, minimizing memory usage.

Disadvantages:

1. Poor Scalability:

From a product perspective, if the product needs an upgrade and additional parameters are involved, the parameters usually cannot pass validation after the upgrade (typically includes length validation), leading to default parameter restoration.

If each module has its own unique structure parameter definitions, adding/deleting parameters will inevitably affect others, causing parameter confusion after device upgrades (the variable addresses in the structure are fixed in the bin file).

2. Poor Readability: If parameters need to be exported, bin files lack readability.

Improvement Measures:

Structures can include reserved definitions, allowing for new parameters to be added in reserved space later, which can somewhat alleviate the scalability issue, meaning new additions won’t affect the size of the existing structure or the positions of other member variables. Deleted parameters can revert to reserved space.

Why is this only a partial solution? Because some modules may not need to add new parameters for a long time or ever, leading to ineffective memory usage, or some modules frequently adding parameters may result in insufficient reserved space. Therefore, careful consideration of reserved allocation during initial design is necessary (after all, memory is limited). The improvements are as follows:

/*****************************
           Before Improvement
*****************************/

typedef struct
{
    uint8_t testParam;
    uint8_t testParam2;
} TestParam_t;    /* Parameters for a certain module */

typedef struct
{
    uint8_t testParam;
    uint8_t testParam2;
    TestParam_t tTestParam;
} SystemParam_t; /* System parameters */

/*****************************
           After Improvement
*****************************/

typedef struct
{
    uint8_t testParam;
    uint8_t testParam2;
    uint8_t reserve[6];    // Reserved
} TestParam_t;    /* Parameters for a certain module */

typedef struct
{
    uint8_t testParam;
    uint8_t testParam2;
    TestParam_t tTestParam;
    uint8_t reserve[50];   // Reserved
} SystemParam_t; /* System parameters */

2.2 JSON Format

Recently, JSON format has become quite popular, especially for data exchange, but it can also be used for parameter storage, using the “{key: value}” format.

Storage Method: String format, i.e., text form

Advantages:

Good Scalability: Due to the JSON format, finding the corresponding key (usually the variable identifier) allows you to find the corresponding value.

Good Readability: With identifiers, exported parameter files can be opened and understood with a regular text editor.

Disadvantages:

Relatively Complex Management: Not as simple as structures; requires learning JSON syntax if unfamiliar.

Higher Memory Usage: Content includes not just values but is stored as strings.

Usage Difficulties: Requires parsing; while C has open-source libraries, they are not convenient due to language nature, whereas C++ is simpler to use.

{
    "SYS":
    {
        "testParam" : 2,
        "testParam2" : 5,
        "tTestParam":
        {
            "testParam" : 2,
            "testParam2" : 5
        }
    }
}

// Compressed string:
{"SYS":{"testParam":2,"testParam2":5,"tTestParam":{"testParam":2,"testParam2":5}}}

2.3 Key-Value Format

Similar to the JSON format, it is also key-value pair format but simpler than JSON.

Storage Method: String format, i.e., text form

Advantages:

Good Scalability: Finding the corresponding key (usually the variable identifier) allows you to find the corresponding value.

Good Readability: With identifiers, exported parameter files can be opened and understood with a regular text editor.

Disadvantages:

Higher Memory Usage: Content includes not just values but is stored as strings.

Slightly Difficult Usage: Requires simple parsing and handling.

Management Issues: Not convenient for managing parameters of various modules according to specific rules.

testParam=2
testParam2=5
T_testParam=2
T_testParam2=5

2.4 Others

There are others, such as XML (similar to JSON), but they will not be discussed further.

3. Compiler Checks for Structure Size and Member Offset

Section 2 introduced three methods for parameter storage; however, for embedded microcontroller development, Flash size is limited, so binary format is usually used for storage. Therefore, this section focuses on solving the scalability issue of managing parameters in structures.

First, let’s discuss pain points (although improvements for scalability have been made, besides the aforementioned issues, there are other pain points that, while not problems, can be critical when they occur).

When adding new parameters to the original reserved space, it is crucial to ensure that the size of the structure remains unchanged; otherwise, it will lead to offsets of other parameters, resulting in abnormal parameters after device upgrades (if a customer upgrades, this can be critical).

To ensure this, every time a new parameter is added, it is necessary to calculate and check whether the size of the structure has changed and whether it has affected other members in the structure.

While manual calculations and checks can catch 99% of issues, people can be careless (working overtime, not in good condition…), and since structures may have padding, one might mistakenly think everything is fine, submit the code, release it (testing may not catch it), deliver it to the customer, and after the upgrade, abnormalities arise, leading to customer complaints and salary deductions (this is tough…).

After encountering such issues: Can’t the compiler check the size or offsets of structure members during compilation? Manual calculations and checks are cumbersome, and mistakes can easily happen. # _ #

In normal circumstances, the compiler doesn’t know the size of the structures you wrote and what you want them to be, so it can’t check (Oh no, this is frustrating 0.0…).

Don’t worry; there are alternative methods to achieve this functionality, allowing the compiler to check for you during compilation with 100% accuracy (of course, when adding new parameters, you still need to reduce the reserved size based on the size of the new parameters, which is necessary).

See the code:

/**
  * @brief Check if the structure size is correct
  *        Will be checked during compilation
  * @param type Structure type
  * @param size Expected size of the structure
  */
#define TYPE_CHECK_SIZE(type, size) extern int sizeof_##type##_is_error [!!(sizeof(type)==(size_t)(size)) - 1]

/**
  * @brief Structure members
  * @param type Structure type
  * @param member Member variable
  */
#define TYPE_MEMBER(type, member) (((type *)0)->member)

/**
  * @brief Check if the structure member size is correct
  *        Will be checked during compilation
  * @param type Structure type
  * @param member Structure member
  * @param size Expected size of the member
  */
#define TYPE_MEMBER_CHECK_SIZE(type, member, size) extern int sizeof_##type##_##member##_is_error \
    [!!(sizeof(TYPE_MEMBER(type, member))==(size_t)(size)) - 1]

/**
  * @brief Check if the size of the structure member of a structure is correct
  *        Will be checked during compilation
  * @param type Structure type
  * @param childtype Child structure type
  * @param member Structure member
  * @param size Expected size of the member
  */
#define TYPE_CHILDTYPE_MEMBER_CHECK_SIZE(type, childtype, member, size) extern int sizeof_##type##_##childtype##_##member##_is_error \
    [!!(sizeof(TYPE_MEMBER(type, childtype.member))==(size_t)(size)) - 1]

/**
  * @brief Check if the offset of a structure member is correct
  *        Will be checked during compilation
  * @param type Structure type
  * @param member Structure member
  * @param value Expected offset
  */
#define TYPE_MEMBER_CHECK_OFFSET(type, member, value) \
         extern int offset_of_##member##_in_##type##_is_error \
        [!!(__builtin_offsetof(type, member)==((size_t)(value))) - 1]

/**
  * @brief Check if the offset of a structure member of a structure is correct
  *        Will be checked during compilation
  * @param type Structure type
  * @param childtype Child structure type
  * @param member Structure member
  * @param value Expected offset
  */
#define TYPE_CHILDTYPE_MEMBER_CHECK_OFFSET(type, childtype, member, value) \
         extern int offset_of_##member##_in_##type##_##childtype##_is_error \
        [!!(__builtin_offsetof(type, childtype.member)==((size_t)(value))) - 1]

With the above code, this problem can be addressed. This method only occupies text size and does not consume memory after compilation!!!

Usage:

typedef struct
{
    uint8_t testParam;
    uint8_t testParam2;
    uint8_t reserve[6];    // Reserved
} TestParam_t;    /* Parameters for a certain module */

TYPE_CHECK_SIZE(TestParam_t, 8); // Check if the structure size meets expectations

typedef struct
{
    uint8_t testParam;
    uint8_t testParam2;
    TestParam_t tTestParam;
    uint8_t reserve[54];   // Reserved
} SystemParam_t; /* System parameters */

TYPE_CHECK_SIZE(SystemParam_t, 64); // Check if the structure size meets expectations
TYPE_MEMBER_CHECK_OFFSET(SystemParam_t, tTestParam, 2); // Check if the offset of member tTestParam meets expectations

Assuming new parameters are added, and the reserved space is incorrectly set, causing the structure size to be incorrect, a compilation error will occur, and the prompt will help quickly locate the issue.