Common Custom Protocol Formats in Embedded Systems

Peripheral Attention+Star Public Account Number to not miss exciting content

Author | strongerHuang

WeChat Public Account | Embedded Column

In embedded development, sensor modules are often used, and these modules typically utilize custom communication protocols.Most custom communication protocols on the market are based on the TLV (Type-Length-Value) format. Let’s discuss the TLV format.

1

TLV Data Format

The TLV (Type-Length-Value) format in embedded IoT is an efficient and flexible data encapsulation method widely used in communication for resource-constrained IoT devices.Structure Analysis:Type identifies the type of data (such as voltage, temperature, humidity, etc.), usually represented by 1-2 bytes. For example, <span><span>0x01</span></span> represents voltage, while <span><span>0x02</span></span> represents temperature. The type essentially determines the byte length of this TLV data unit and the parsing method for the Value.Length indicates the byte count of the Value field, which can be fixed or variable length. For example, 1 byte (0-255) or variable-length integers (to save space).Value is the actual data content, with the length defined by the Length field, such as a temperature value<span><span>3.11V, amplified 1000 times encoded as an integer of two bytes</span></span>0x0C 0x1C.

Having introduced the TLV data encapsulation format, let’s detail its three main characteristics:

2

Self-describing Type

The first part of each TLV structure is the Type field (usually 1-4 bytes), which clearly identifies the semantics and encoding of the data, as mentioned earlier.

However, this field needs to be agreed upon by both the sender and receiver, leading to the creation of a Type Mapping Table.

The Type Mapping Table: The Type value is determined through predefined protocol specifications or dynamic negotiation (such as during device handshake), and the receiver selects the corresponding parsing logic based on the Type value. If the Type value does not match the actual data format (for example, if Type declares an integer but Value is a string), the receiver can directly report an error.

This way, no external metadata is needed; the data itself carries type information, and the parser does not need to rely on additional protocol documents or configuration files. Many advanced serialization tools require such tools, which can be cumbersome.

In terms of compatibility, different devices or protocol versions can achieve compatibility by extending Type definitions (such as reserving private Type ranges), allowing old devices to ignore unknown Types. Similarly, when upgrading the protocol, you only need to define new Type values (for example, 0x04 represents PM2.5), allowing old devices to skip unknown fields while new devices parse new data, provided you define a sufficient number of type fields.

3

Dynamic Variable Length

The length of the Length field following the Type is not fixed, usually 1-4 bytes is sufficient, and it primarily declares the byte length of the Value part.

For example:

If Value is a 4-byte integer, Length=0x04

If Value is a variable-length string "Hello", Length=0x05

Of course, more flexible encoding can be used, and the byte count of the Length field can be dynamically adjusted according to protocol design.

Short data (≤255 bytes): Length is represented by 1 byte;

Long data (such as firmware packages): Length is represented by 4 bytes

This not only saves space by avoiding padding waste from fixed-length fields but also supports variable data to meet the changing data length requirements in IoT scenarios (such as sensor data, configuration parameters, log text).

Have you ever used streaming transmission? By knowing the data boundary in advance through Length, there is no need to wait for a complete data packet, allowing the receiver to parse data while receiving.

4

Efficient Binary Encoding

The TLV format is transmitted as a binary byte stream rather than a text format (such as JSON, XML). Moreover, a compact encoding rule is adopted in the protocol specification, such as:

Integers use the minimum number of bytes (for example, 1 byte for values within 0x7F, 2 bytes for values exceeding that);

Floating-point numbers are compressed using the IEEE 754 standard;

Enumeration values are directly represented by a single byte, etc., allowing for direct byte-offset reading of data without conversion, which is indeed favored on resource-constrained platforms. Custom protocols based on TLV data encapsulation can basically meet most embedded application scenarios.

———— END ————● Column “Embedded Tools”● Column “Embedded Development”● Column “Keil Tutorials”● Selected Tutorials from the Embedded ColumnFollow the public accountReply “Join Group” to join the technical exchange group according to the rules, reply “1024” to see more content.Click “Read the Original” to see more shares.