Segmentation Offloads

Introduction

This document introduces a set of techniques in the Linux networking stack that leverage the segmentation offload capabilities of various network interface cards.

The following techniques will be discussed:

TCP Segmentation Offload – TSO (TCP Segmentation Offload)
UDP Fragmentation Offload – UFO (UDP Fragmentation Offload)
IPIP, SIT, GRE, and UDP Tunnel Offload
Generic Segmentation Offload – GSO (Generic Segmentation Offload)
Generic Receive Offload – GRO (Generic Receive Offload)
Partial Generic Segmentation Offload – GSO_PARTIAL
SCTP acceleration using GSO – GSO_BY_FRAGS

TCP Segmentation Offload

TCP segmentation allows (network) devices to split a single frame into multiple frames, with the data payload size specified by skb_shinfo()->gso_size. When TCP segmentation is requested, the skb_shinfo()->gso_type should be set to SKB_GSO_TCPV4 or SKB_GSO_TCPV6, and skb_shinfo()->gso_size should be set to a non-zero value.

TCP segmentation relies on support for partial checksum offload. Therefore, if a (network) device’s Tx checksum offload capability is disabled, TSO is typically also disabled.

To support TCP segmentation offload, the network and transport header offsets of the socket buffer (skbuff) must be populated so that the device driver can determine the offsets of the IP or IPv6 header and the TCP header. Additionally, since CHECKSUM_PARTIAL is required, csum_start should also point to the TCP header of the packet.

For IPv4 segmentation, we support one of two types regarding the IP identification (IP ID). The default behavior is to increment the IP ID for each segment. If the GSO type SKB_GSO_TCP_FIXEDID is specified, the IP ID will not be incremented, and all segments will use the same IP ID. If the device is set with NETIF_F_TSO_MANGLEID, the IP ID can be ignored during TSO, and we will increment the IP ID for all frames or keep it static based on the driver’s preference.

UDP Fragmentation Offload

UDP fragmentation offload allows devices to split oversized UDP datagrams into multiple IPv4 fragments. Many of the requirements for UDP fragmentation offload are the same as for TSO. However, since a single IPv4 datagram is fragmented, the IPv4 ID of the fragments should not be incremented.

UFO has been deprecated: Modern kernels will no longer generate UFO socket buffers (skbs), but they can still be received from Tuntap and similar devices. Offload for UDP-based tunneling protocols is still supported.

IPIP, SIT, GRE, UDP Tunnels, and Remote Checksum Offload

In addition to the offload capabilities mentioned above, a frame may also contain additional headers, such as external tunnel headers. To handle such cases, a set of additional segmentation offload types has been introduced, including SKB_GSO_IPXIP4, SKB_GSO_IPXIP6, SKB_GSO_GRE, and SKB_GSO_UDP_TUNNEL. These additional segmentation types are used to identify cases where multiple sets of headers exist. For example, in the case of IPIP and SIT, we should move the network and transport headers from the standard header list to the “inner” header offsets.

Currently, only two-level headers are supported. Typically, we refer to the tunnel header as the external header, while the encapsulated data is referred to as the internal header. Here is a list of calls to access a given header:

IPIP/SIT Tunnel:

           External                  Internal
MAC       skb_mac_header
Network    skb_network_header     skb_inner_network_header
Transport    skb_transport_header

UDP/GRE Tunnel:

           External                  Internal
MAC       skb_mac_header         skb_inner_mac_header
Network    skb_network_header     skb_inner_network_header
Transport    skb_transport_header   skb_inner_transport_header

In addition to the tunnel types mentioned above, there are also SKB_GSO_GRE_CSUM and SKB_GSO_UDP_TUNNEL_CSUM. These two additional tunnel types reflect cases where the external header also requires a non-zero checksum to be included.

Finally, there is SKB_GSO_TUNNEL_REMCSUM, which indicates that the given tunnel header requests remote checksum offload. In this case, the internal header will retain a partial checksum, and only the external header’s checksum will be calculated.

Generic Segmentation Offload

Generic Segmentation Offload is a pure software offload designed to handle cases where the device driver cannot perform the aforementioned offload functions. In GSO, the data of the given socket buffer (skbuff) will be split into multiple socket buffers, with their sizes adjusted to match the maximum segment size (MSS) provided by skb_shinfo()->gso_size.

Before enabling any hardware segmentation offload, the corresponding software offload must be enabled in GSO. Otherwise, a frame may be rerouted between devices and ultimately fail to be transmitted.

Generic Receive Offload

Generic Receive Offload complements GSO. Ideally, any frame assembled by GRO should be able to be segmented using GSO to create the same frame sequence, and any frame sequence segmented by GSO should be able to be reassembled back into the original frame by GRO. The only exception is when the DF bit is set in the given IP header for the IPv4 ID. If the value of the IPv4 ID is not sequentially incrementing, then when frames assembled by GRO are segmented by GSO, it will be modified to be sequentially incrementing.

Partial Generic Segmentation Offload

Partial Generic Segmentation Offload is a hybrid scheme of TCP Segmentation Offload (TSO) and Generic Segmentation Offload (GSO). Its actual effect is to leverage certain characteristics of TCP and tunnels so that when rewriting the packet headers for each segment, only the innermost transport layer header needs to be updated, and possibly the outermost network layer header as well. This allows devices that do not support tunnel offload or offload with checksums to still utilize segmentation functionality.

During partial offload, all headers except for the innermost transport layer header will be updated so that these headers still contain the correct values in the case of simple copying. The only exception is the outer IPv4 identification (ID) field. If the given header does not have the Don’t Fragment (DF) bit set, the device driver is responsible for ensuring that the IPv4 ID field is incrementing.

SCTP Acceleration Using GSO

Despite the lack of hardware support, the Stream Control Transmission Protocol (SCTP) can still leverage Generic Segmentation Offload (GSO) to send a large packet through the network stack instead of multiple small packets.

This requires a different approach than other offload methods, as SCTP packets cannot simply be segmented to the (Path) Maximum Transmission Unit ((P)MTU) size. Instead, data chunks must be included in IP segments, and padding bytes must be considered. Therefore, unlike regular GSO, SCTP cannot simply generate a large socket buffer (skb), set gso_size to the segmentation point, and then deliver it to the IP layer.

Instead, the SCTP protocol layer will construct a socket buffer (skb) where the segments are correctly padded and stored as a chain of socket buffers, and then the skb_segment() function will segment based on these. To indicate this, <code>gso_size will be set to a special value GSO_BY_FRAGS.

Thus, any code in the network core stack must consider the case where gso_size may be GSO_BY_FRAGS and handle it appropriately.

There are some helper functions that can simplify this process:

skb_is_gso(skb) && skb_is_gso_sctp(skb) is the best way to determine if a socket buffer (skb) is an SCTP GSO socket buffer.
For size checks, the skb_gso_validate_*_len series of helper functions can correctly handle the GSO_BY_FRAGS case.
For manipulating packets, skb_increase_gso_size and skb_decrease_gso_size will check if it is GSO_BY_FRAGS, and will issue a warning if an attempt is made to manipulate such socket buffers.

This will also affect drivers that have the NETIF_F_FRAGLIST & NETIF_F_GSO_SCTP flags set. Additionally, note that NETIF_F_GSO_SCTP is included in NETIF_F_GSO_SOFTWARE.

Source

https://docs.kernel.org/networking/segmentation-offloads.html

Segmentation Offloads in Linux Networking