Network Protocols
When we connect to the internet using our mobile phones, we use many network protocols. Starting from connecting the phone to WiFi, the 802.11 (i.e., WLAN) protocol is used to access the network; the phone automatically obtains network configuration using the DHCP protocol, and only after obtaining the configuration can the phone communicate normally. At this point, the phone is already connected to a local area network and can access devices and resources within the local area network, but it cannot use internet applications such as WeChat or Douyin. To access the internet, it is necessary to implement relevant protocols on the upstream network device of the phone, that is, to configure NAT, PPPOE, etc., on the wireless router, and then connect the local area network to the internet via the internet line provided by the operator, allowing the phone to use WeChat and browse Douyin.
Local Area Network: A private network within a small range, such as a home network, a company network, or a campus network.
Wide Area Network: A network that connects local area networks in different regions. Operators build wide area networks to achieve interconnection across regions.
Internet: A network that connects the entire world. The internet is an open and interconnected network that does not belong to any individual or organization, and once connected to the internet, it can communicate with any host on the internet.
In simple terms, devices such as mobile phones and wireless routers communicate through various network protocols. Network protocols are standards or rules defined for communication parties to communicate with each other. As long as devices adhere to the same network protocol, communication can be achieved. But who defines the network protocols? The ISO has established an international standard called OSI, and the OSI reference model is often used for the formulation of network protocols.
OSI Reference Model
The OSI reference model divides the services provided by network protocols into 7 layers and defines the service content of each layer. The services of each layer are implemented by protocols, and the specific content of the protocols is rules. The upper and lower layers interact through interfaces, and the same layer interacts through protocols. The OSI reference model only provides a rough definition of the services of each layer and does not define protocols in detail, but many protocols correspond to a certain layer of the 7-layer structure. Therefore, to understand the network, one must first understand the OSI reference model.
Application Layer
The 7th layer (top layer) of the OSI reference model. It serves as the interface between applications and the network, providing services directly to users. Application layer protocols include email, remote login, and others.
Presentation Layer
The 6th layer of the OSI reference model. Responsible for the conversion of data formats, such as encoding, data format conversion, and encryption/decryption. Ensures that information sent from the application layer of one system can be read by the application layer of another system.
Session Layer
The 5th layer of the OSI reference model. Mainly manages and coordinates communication (dialogue) between various processes on different hosts, that is, responsible for establishing, managing, and terminating sessions between applications.
Transport Layer
The 4th layer of the OSI reference model. Provides reliable and transparent data transmission services between communication hosts for upper layer protocols, including error control and flow control, among other issues. This is processed only on communication hosts and does not need to be processed on routers.
Network Layer
The 3rd layer of the OSI reference model. Responsible for transmitting data to the destination address on the network, mainly responsible for addressing and routing.
Data Link Layer
The 2nd layer of the OSI reference model. Responsible for communication transmission between two interconnected hosts at the physical layer, dividing the bit stream composed of 0s and 1s into data frames for transmission to the other end, that is, the generation and reception of data frames. Communication transmission is actually achieved through physical transmission media. The data link layer’s role is to process data between these devices interconnected through transmission media.
Both the network layer and the data link layer send data to the receiving end based on the target address, but the network layer is responsible for sending the entire data to the final target address, while the data link layer is only responsible for sending data within a segment.
Physical Layer
The 1st layer of the OSI reference model (the lowest layer). Responsible for the conversion between logical signals (bit streams) and physical signals (electrical signals, optical signals), providing a physical connection for the data link layer through transmission media.
TCP/IP Reference Model
Because the OSI reference model divides services too granularly, defining the reference model first and then defining protocols is somewhat idealistic. The TCP/IP model, on the other hand, is summarized from existing protocols, becoming the practical network protocol standard in the industry.
TCP/IP is a protocol suite suggested and promoted for standardization by the IETF, consisting of protocols such as IP, TCP, HTTP, etc. TCP/IP is the protocol family developed for use on the internet, so the internet protocol is TCP/IP.
First, let’s introduce the correspondence between TCP/IP and OSI layers, as well as the main protocols at each layer of TCP/IP.
Network Access Layer
TCP/IP is formulated on the premise that the functions of the physical layer and data link layer of the OSI reference model are transparent and does not define these two layers, so the physical layer and data link layer can be combined and referred to as network access layer. The network access layer manages the network medium, defining how to use the network to transmit data. However, during communication, these two layers serve different purposes, so they can also be referred to as hardware and network interface layer. TCP/IP can be divided into four or five layers as long as the principles are understood.
Devices are interconnected through physical transmission media, and the data transfer between interconnected devices is achieved using MAC addresses. The purpose of using MAC addresses is to identify devices connected to the same transmission medium.
Network Layer
Corresponding to the 3rd layer of the OSI model, using the IP protocol. The IP protocol forwards packet data based on IP addresses, and its role is to send data packets from the source address to the destination address.
The functions of the network layer and transport layer in the TCP/IP layering are usually provided by the operating system. Routers achieve the function of forwarding data packets through the network layer.
During network transmission, each node will determine which network card should send the message based on the address information of the data. Each address will refer to a list of outgoing interfaces, and the table referenced in MAC addressing is called the MAC address table, while the table referenced in IP addressing is called the routing table. The MAC address table is automatically generated based on self-learning. The routing table is automatically generated based on routing protocols. The MAC address table records the actual MAC addresses, while the routing table records the IP addresses, which are the aggregated network numbers (i.e., network number and subnet mask).
-
IP
IP is a protocol for transmitting data packets across networks, using IP addresses as host identifiers, allowing the entire internet to receive data. The IP protocol is independent of the underlying medium, achieving data forwarding from source to destination. The IP protocol does not have a retransmission mechanism and is considered an unreliable transmission protocol.
-
ICMP
Used to transmit control messages between IP hosts and routers, used to diagnose the health of the network.
-
ARP
A protocol that resolves the MAC address from the IP address of a data packet.
Transport Layer
Corresponding to the 4th layer of the OSI model, the main function is to enable communication between applications by identifying applications using port numbers, with protocols including connection-oriented TCP protocol and connectionless UDP protocol.
Connection-oriented means establishing a logical communication link between the sending and receiving hosts before sending data. It’s like making a phone call; after dialing the other party’s number, you can only talk when the other party answers the phone, and hanging up is like cutting off the power.
Connectionless does not require establishing and breaking a connection. The sender can freely send data at any time. It’s like sending a letter; you do not need to confirm whether the recipient’s information is real or whether the recipient can receive the letter, as long as there is a mailing address.
-
TCP
TCP is a connection-oriented transport layer protocol that can control the connection it provides. It is suitable for applications requiring reliable transmission, such as file transfers.
-
UDP
UDP is a connectionless transport layer protocol that does not control the connection it provides. It is suitable for real-time applications, such as IP telephony, video conferencing, live streaming, etc.
Application Layer
Corresponding to the combination of the 5th to 7th layers of the OSI model, it not only implements the functions of the application layer of the OSI model but also the functions of the session layer and presentation layer. HTTP, POP3, TELNET, SSH, FTP, SNMP are all application layer protocols.
Most TCP/IP applications belong to the client/server model. The program providing the service is called the server, while the program receiving the service is called the client. The client can send requests to the server at any time.
-
HTTP
It is the application layer communication protocol between WWW browsers and servers, with the main data format being HTML. HTTP defines high-level commands or methods for browsers to communicate with web servers.
-
POP3
Simple Mail Transfer Protocol, used between mail clients and mail servers.
-
TELNET and SSH
Remote terminal protocols used for remotely managing network devices. TELNET transmits in plaintext, while SSH transmits encrypted.
-
SNMP
Simple Network Management Protocol, used for network management software to monitor and manage network devices.
Encapsulation and Decapsulation
Typically, the information provided for protocols is the header of the packet, and the content to be sent is the data. In each layer, a header containing necessary information, such as the destination address and protocol-related information, is added to the data being sent. From the perspective of the next layer, all packets received from the previous layer are regarded as data of that layer.
Before sending data, according to the reference model from top to bottom, when the data passes through each layer, protocol message header information is added; this process is called encapsulation.
After receiving data, according to the reference model from bottom to top, when the data passes through each layer, protocol header information is removed; this process is called decapsulation.
Data encapsulated by the transport layer protocol is called a segment, data encapsulated by the network layer protocol is called a packet, data encapsulated by the data link layer protocol is called a frame, and data transmitted by the physical layer is called bit.
In TCP/IP communication, MAC addresses, IP addresses, port numbers, and other information are used as address identifiers. Even at the application layer, an email address can be used as the address for network communication.
Example of Actual Data Transmission
In real life, the internet uses the TCP/IP protocol for network connections. Let’s take visiting a website as an example to see how the network communicates.
Sending Data Packets
When visiting an HTTP website, open the browser, enter the URL, and press Enter to begin TCP/IP communication.
Application Processing
First, the application will perform HTML format encoding, which is equivalent to the function of the OSI presentation layer. After encoding, the data may not be sent immediately, which corresponds to the function of the session layer. At the moment the request is sent, a TCP connection is established, and then data is sent over the TCP connection. The data is then sent to the next layer, TCP, for processing.
TCP Module Processing
TCP will smoothly send the data received from the application layer to the destination. To achieve reliable transmission, it is necessary to encapsulate the data with TCP header information. The TCP header information includes source port number and destination port number (to identify applications on the host), sequence number (to confirm which part is data), and checksum (to determine if the data has been corrupted). The segment encapsulated with TCP header information is then sent to IP.
IP Module Processing
IP treats the segment sent by TCP as its own data and encapsulates it with IP header information. The IP header information includes destination IP address and source IP address, as well as the upper layer protocol type information.
After the IP packet is generated, it is sent based on the host’s routing table.
Network Interface Processing
The network interface encapsulates the incoming IP packet with Ethernet header information and processes it for sending. The Ethernet header information includes destination MAC address, source MAC address, and upper layer protocol type information. The Ethernet data frame is then transmitted to the receiving end through the physical layer. The FCS in the sending process is calculated by hardware and added to the end of the packet. The purpose of setting FCS is to determine whether the data packet has been corrupted due to noise.
Receiving Data Packets
The packet receiving process is the reverse of the sending process.
Network Interface Processing
After receiving the Ethernet packet, it first checks whether the destination MAC address in the header information is the intended recipient. If not, the packet is discarded. If it is the intended recipient, it checks whether the upper layer protocol type is an IP packet; the Ethernet frame is decapsulated into an IP packet and passed to the IP module for processing. If the protocol type is unrecognized, the data is discarded.
IP Module Processing
After receiving the IP packet, similar processing is performed. Based on the destination IP address in the header information, it determines whether it is the intended recipient. If it is, it checks the upper layer protocol type. If the upper layer protocol is TCP, the IP packet is decapsulated and sent to the TCP protocol for processing.
If there is a router and the receiving end is not its own address, the data is forwarded according to the routing control table.
TCP Module Processing
After receiving the TCP segment, it first checks the checksum to determine if the data has been corrupted. Then, it checks whether the data has been received in the correct order. Finally, it checks the port number to identify the specific application.
After the data is completely received, an “acknowledgment receipt” is sent to the sender. If this acknowledgment does not reach the sender, the sender will assume that the receiver has not received the data and will repeatedly send it.
Once the data is completely received, the TCP segment is decapsulated and sent to the application identified by the port number.
Application Processing
The application receives the data and parses the content to determine the requested webpage content, then performs subsequent data exchanges according to the HTTP protocol.
Network Composition
Building a network involves various cables and network devices. Below are some common hardware devices. The layer numbers of hardware devices refer to the OSI reference model, not the TCP/IP model.
Transmission Medium and Data Link
Devices are connected via cables. Wired cables include twisted pairs, fiber optics, serial cables, etc. Choose the corresponding cable based on the data link. Transmission media can also be divided into different types of electromagnetic waves, such as radio waves and microwaves.
Transmission Rate: Measured in bps, which indicates how much data is transmitted in a unit of time. Also known as bandwidth, the larger the bandwidth, the stronger the network transmission capacity.
Throughput: Measured in bps, representing the actual transmission rate between hosts. The term throughput not only measures bandwidth but also considers the CPU processing capability of the hosts, network congestion, and the share of data fields in the message.
Network Interface Card (NIC)
Any host must use a network interface card to connect to the network. It can be a wired NIC for connecting to a wired network or a wireless NIC for connecting to a WiFi network. Each NIC has a unique MAC address, also called hardware address or physical address.
Layer 2 Switch
Layer 2 switches operate at the 2nd layer (data link layer) of the OSI model. They can recognize data frames in the data link layer and forward frames to another data link.
A data frame contains a data bit called FCS, which is used to verify whether the data has been correctly delivered to the destination. Layer 2 switches check this value to discard corrupted data.
Layer 2 switches use a self-learning mechanism based on MAC addresses to determine whether to forward data frames.
Router / Layer 3 Switch
Routers operate at the 3rd layer (network layer) of the OSI model, connecting two networks and forwarding packets. While layer 2 switches process based on MAC addresses, routers / layer 3 switches process based on IP addresses. Therefore, the address at the network layer in TCP/IP becomes the IP address.
Routers can connect different data links. For example, they can connect two Ethernet networks or connect an Ethernet network to a wireless network. Common household wireless routers are a type of router.
Layer 4-7 Switches
Layer 4-7 switches are responsible for processing data from the transport layer to the application layer in the OSI model. Based on TCP and other transport layer protocols, they analyze the data being sent and received and perform specific processing. For instance, if a server on a video website cannot meet the access demand, a load balancing device distributes the access across multiple backend servers, which is one application of layer 4-7 switches. Other application scenarios include bandwidth control, wide area network accelerators, firewalls, etc.
Conclusion
Application layer devices include computers, mobile phones, servers, etc. Application layer devices do not forward data; they are sources or destinations of data and possess functions of all layers below the application layer. When sending data, it encapsulates the data layer by layer from top to bottom before sending it out through Ethernet. When receiving data, it decapsulates the data layer by layer from bottom to top, ultimately restoring it to its original form.
Data link layer devices include layer 2 switches, bridges, etc. Layer 2 network devices only forward data, forwarding it by recognizing the MAC addresses. After receiving data, the layer 2 switch checks the Ethernet header information of the outermost layer of the data, sees the destination MAC address, and sends the data frame out from the corresponding port. The switch does not decapsulate the data frame; it only needs to know the MAC address information to forward the data correctly.
Network layer devices include routers, layer 3 switches, etc. Layer 3 network devices only forward data, forwarding it by recognizing the IP addresses. After receiving data, the router first checks the outermost Ethernet header information; if the destination MAC address is itself, it will decapsulate the Ethernet header and look at the IP address of the data. When making forwarding decisions based on the IP routing table, the router takes the MAC address of the next hop device as the destination MAC address of the Ethernet header, re-encapsulates the Ethernet header, and forwards the data.
The network devices that forward data and the data at the application layer are like couriers and packages. Couriers deliver packages based on the destination address without needing to understand the contents of the packages.
Distinguishing network devices based on layered functions is no longer applicable; switches with integrated layer 3 routing functions are called layer 3 switches, and wireless APs integrated with router functions are called wireless routers. However, for convenience of explanation, individual device functions and principles are typically described.
References:
Illustrated TCP/IP – Takashi Takeshita
Networking Basics – Tian Guo
end
Exciting Recommendations for Getting Started in Learning Networking Technologies
Illustrated Networking PDF
123 Diagrams Explaining 177 Computer Networking Terms
Series of Illustrated Networking
Switching and Routing
Interesting Knowledge