-
Network Card to Memory
-
Kernel’s Network Module
-
Protocol Stack
-
IP Layer
-
UDP Layer
-
Socket
-
Conclusion
This article will introduce how a UDP packet is received step by step from the network card to the process in a Linux system.
Network Card to Memory
The network interface card must have the appropriate driver installed to function correctly. These drivers are considered kernel modules, and their main responsibility is to connect the network card and the kernel’s network module. When the driver is loaded, it registers itself with the network module. When the corresponding network card receives a packet, the network module calls the appropriate driver to handle the data.
The following diagram shows how a packet enters memory and is processed by the kernel’s network module:
-
1: The incoming packet from the external network enters the physical network card. If the destination address does not belong to this network card and the card is not in promiscuous mode, the packet will be discarded by the card. -
2: The network card uses Direct Memory Access (DMA) technology to write the packet to a designated memory address. These memory addresses are allocated and initialized by the network card driver. -
3: The network card sends a notification to the CPU via a hardware interrupt request (IRQ) to inform that data has arrived. -
4: The CPU calls the registered interrupt handler function based on the interrupt table configuration, which further calls the corresponding function in the network card driver (network interface card driver). -
5: The driver first disables the card’s interrupt functionality, indicating that the driver is aware that data has been stored in memory and informs the network card to write directly to memory when the next packet is received, without notifying the CPU again, thus improving efficiency and avoiding frequent CPU interruptions. -
6: A soft interrupt is initiated. The hard interrupt handler cannot be interrupted while executing; if it takes too long, the CPU will be unable to respond to other hardware interrupts. Therefore, the kernel introduces the concept of soft interrupts, moving the time-consuming parts of the hard interrupt handler to the soft interrupt handler for gradual processing.
Kernel’s Network Module
The soft interrupt triggers the soft interrupt handler in the kernel’s network module, and the subsequent process is as follows:
-
7: In the operating system kernel, there is a dedicated process for handling soft interrupts called ksoftirqd. When ksoftirqd receives a soft interrupt, it calls the corresponding soft interrupt handler function. For the soft interrupt triggered by the network card driver module mentioned in step 6, ksoftirqd calls the net_rx_action function in the network module. -
8: The net_rx_action function calls the poll function in the network card driver to process packets one by one. -
9: In the poll function, the driver reads the packets written to memory by the network card, the format of which is known only to the driver. -
10: The driver converts the packet in memory to the skb format recognizable by the kernel’s network module and calls the napi_gro_receive function. -
11: The napi_gro_receive function handles tasks related to GRO (Generic Receive Offload), merging packets that can be combined, thus calling the protocol stack only once. It then checks if RPS (Receive Packet Steering) is enabled; if so, it calls the enqueue_to_backlog function. -
12: In the enqueue_to_backlog function, the packet is placed in the input_pkt_queue of the CPU’s softnet_data structure and then returned. If the input_pkt_queue is full, the packet will be discarded; the size of this queue can be configured via the net.core.netdev_max_backlog parameter. -
13: The CPU will process the network data in the input_pkt_queue in its soft interrupt context (calling __netif_receive_skb_core function). -
14: If RPS is not enabled, the napi_gro_receive function calls the __netif_receive_skb_core function directly. -
15: First, it checks for the presence of AF_PACKET type sockets (i.e., raw sockets); if present, the packet is copied to that socket. For example, the packets captured by tcpdump are caught here. -
16: The relevant protocol stack function is called to hand the packet over for processing. -
17: After all packets in memory have been processed (i.e., the poll function completes), the network card’s hard interrupt is enabled so that when the network card receives the next batch of data, it will notify the CPU.
The enqueue_to_backlog function is also called by the netif_rx function, which is the function called when the lo device sends packets.
Protocol Stack
IP Layer
Since this is a UDP packet, the first step will enter the IP layer, and then functions will be called down one by one:
-
ip_rcv: The ip_rcv function is the entry function for the IP module. The first thing it does is discard garbage packets (packets whose destination MAC address does not belong to the current network card but were received due to the card being in promiscuous mode) and then calls functions registered on NF_INET_PRE_ROUTING. -
NF_INET_PRE_ROUTING: Netfilter hooks placed in the protocol stack that can inject some packet processing functions through iptables to modify or discard packets. If the packet is not discarded, it will continue down the line. -
routing: Routing occurs; if the destination IP is not the local IP and IP forwarding is not enabled, the packet will be discarded. If IP forwarding is enabled, it will enter the ip_forward function. -
ip_forward: The ip_forward function first calls the NF_INET_FORWARD related functions registered with netfilter. If the packet is not discarded, it will continue to call the dst_output_sk function. -
dst_output_sk: This function will call the appropriate function in the IP layer to send the packet out. -
ip_local_deliver: If the routing process above finds that the destination IP is the local IP, this function will be called. In this function, the NF_INET_LOCAL_IN related hooks will be called first. If passed, the packet will be sent down to the UDP layer.
UDP Layer
-
The udp_rcv function is the entry function for the UDP module, used to handle received UDP packets. In this function, a series of checks are performed, and other functions are called for processing. An important function call is __udp4_lib_lookup_skb, which looks up the corresponding socket based on the target IP and port. If no corresponding socket is found, the packet will be discarded; otherwise, processing will continue. -
The sock_queue_rcv_skb function primarily performs two checks. First, it checks if the socket’s receive buffer is full; if it is, the packet will be discarded. Then, it calls the sk_filter function to check if the packet meets the filter conditions set on the current socket. If filtering conditions are set on the socket and the packet does not meet them, the packet will also be discarded. In Linux, each socket can define filter conditions like in tcpdump, and packets that do not meet the conditions will be discarded. -
The __skb_queue_tail function is used to place the packet at the end of the socket’s receive queue. -
The sk_data_ready function is used to notify the socket that the packet is ready to be processed.
After calling sk_data_ready, a packet’s processing is complete, waiting for the application layer program to read it. The entire execution process of the above functions occurs in the context of a soft interrupt.
Socket
There are generally two ways for the application layer to receive data: one is the recvfrom function blocking while waiting for data; in this case, when the socket receives a notification, recvfrom will be awakened and then read the data from the receive queue; the other is through epoll or select to listen to the corresponding socket, and after receiving a notification, call the recvfrom function to read the data from the receive queue. Both methods can successfully receive the corresponding packets.
Conclusion
Understanding the packet reception process helps us clarify where we can monitor and modify packets, under what circumstances packets may be discarded, and provides some references for handling network issues. Additionally, understanding the relevant hook positions in netfilter is helpful for understanding the usage of iptables.