Notes on Understanding HTTP

❝

This is the 285th article in the series on growth by Guoxing.

❞

(1) Reading Understanding HTTP

The duration is three weeks from July 11 to July 31.

1. Why Learn HTTP?

The front end (client) displays pages and interactions;

The back end (server, logic code, data) completes function writing and saves persistent data;

So how do the front end pages and back end achieve interaction?

One important way to achieve this is by using HTTP.

In addition,

• Writing web crawler programs
• Analyzing packet capture data
• Implementing HTTP servers
• Providing website REST APIs
• Modifying backend custom frameworks, etc.

They all, without exception, require knowledge of various aspects of the HTTP protocol, and both front-end and back-end developers need to learn HTTP.

However, HTTP itself can be quite complex and tedious, so to make it easier for everyone to get started, we chose Understanding HTTP.

2. The Content of This Book Includes:

• The first half is the history of the growth and development of HTTP;
• Based on the HTTP 1.1 standard, it explains the communication process, including the specific meanings of HTTP methods, protocol formats, message structures, header fields, status codes, etc., and explains the roles of proxies, gateways, tunnels, etc. during HTTP communication;
• Next, it introduces the extended features of HTTP such as SPDY, WebSocket, WebDAV;
• Details such as the difference between stateless, 301 and 302 redirects, caching mechanisms, etc.;
• The second half focuses on web security: covering HTTPS, SSL, certificate authentication, encryption mechanisms, web attack methods, etc.

3. Suitable Readers

• Developers of web applications
• Software developers using the web
• Security engineers for web risk assessment
• Front-end engineers
• All readers related to the web, such as web users

(2) Web and Network Basics

1. Client

❝

Web browsers, which send requests to obtain server resources, can all be referred to as clients.

❞

2. HTTP Related Terms and Full Names

• HTTP (HyperText Transfer Protocol, 超文本传输协议)
• WWW (World Wide Web, 万维网)
• HTML (HyperText Markup Language, 超文本标记语言)
• URL (Uniform Resource Locator, 统一资源定位符)

3. Version History

(1) HTTP/1.0

❝

HTTP was officially published as a standard in May 1996, and the version was named HTTP/1.0, recorded in RFC1945. Although it is an early standard, this protocol standard is still widely used on the server side today.

❞

(2) HTTP/1.1

❝

HTTP/1.1, published in January 1997, is the current mainstream version of the HTTP protocol. The initial standard was RFC2068, and the subsequent revision RFC2616 is the current latest version.

❞

(3) HTTP/2.0

❝

HTTP/2 was developed by the IETF HTTP/2 working group, approved as a formal specification on February 17, 2015, and RFC 7540 was published in May 2015. HTTP/2 is the first version upgrade in 16 years since HTTP/1.1 was designated as RFC 2616 in June 1999.

❞

(4) HTTP/3.0

On June 6, 2022, IETF officially standardized HTTP/3 as RFC9114.

❝

Unlike its predecessors HTTP/1.1 and HTTP/2, HTTP/3 will abandon the TCP protocol and instead use the QUIC protocol based on UDP. This change is mainly to solve the head-of-line blocking problem existing in HTTP/2. Because HTTP/2 uses multiplexing on a single TCP connection, it is affected by TCP congestion control, and a small amount of packet loss can cause all streams on the entire TCP connection to be blocked.

❞

(3) The Upstream and Downstream of HTTP

1. TCP/IP Protocol Suite

• Application Layer

  ❝

  FTP (File Transfer Protocol) File Transfer
  DNS (Domain Name System) Domain Name System
  HTTP

  ❞

• Definition: The application layer determines the communication activities when providing application services to users.
• Classification
• Transport Layer

  ❝

  TCP (Transmission Control Protocol) Transmission Control Protocol
  UDP (User Datagram Protocol) User Datagram Protocol

  ❞

• Definition: Provides data transmission between two computers in a network connection to the upper application layer.
• Classification
• Network Layer
• Function: Used to handle data packets flowing on the network.
• Definition: This layer specifies how data reaches the other computer.
• Link Layer
• Function: Used to handle the hardware part of the network connection.
• Definition: Includes controlling the operating system, hardware device drivers, NIC (network interface card), and other physically visible parts.

❝

When the sender transmits data between layers, each layer will definitely add a header information belonging to that layer. Conversely, when the receiver transmits data between layers, it will remove the corresponding header at each layer.

❞

2. IP Protocol

IP (Internet Protocol) is located at the network layer, and “IP” is actually the name of a protocol.

❝

Function: The role of the IP protocol is to deliver various data packets to the other party.

❞

• The IP address specifies the address assigned to the node.
• The MAC address is the fixed address of the network card.

# How to check Mac's IP and Mac address
ifconfig
# en0  IP address is 192.168.1.8  MAC address is 14:7d:da:ab:8f:51
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
 options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
 ether 14:7d:da:ab:8f:51
 inet6 fe80::48c:77a5:931c:4df9%en0 prefixlen 64 secured scopeid 0x6
 inet 192.168.1.8 netmask 0xffffff00 broadcast 192.168.1.255
 inet6 2409:8a00:2444:5c20:1405:aa90:6b2f:3c00 prefixlen 64 autoconf secured
 inet6 2409:8a00:2444:5c20:b1af:308c:5fda:ab8c prefixlen 64 autoconf temporary
 nd6 options=201<PERFORMNUD,DAD>
 media: autoselect
 status: active

3. Ensuring Reliability with TCP Protocol

Hierarchically, TCP is located at the transport layer and provides reliable byte stream service.

❝

The so-called byte stream service refers to managing large blocks of data by dividing them into data packets managed by segments.

❞

The TCP protocol divides the data to make it easier to transmit large data, and it can confirm whether the data is ultimately delivered to the other party.

To accurately deliver data to the target, the TCP protocol uses a three-way handshake strategy.

❝

The handshake process uses TCP flags — SYN (synchronize) and ACK (acknowledgement).

• SYN (synchronize) Synchronizer

❞

• ACK (acknowledgement) Confirmation

4. Various Protocols and Their Relationship with HTTP Protocol

Like a hamburger

• In the middle is the IP protocol for point-to-point transmission via routers
• The outer layer is the TCP protocol, which splits into trusted segments, and after transmission, it is reassembled
• The outermost layer is the HTTP protocol, which requests and parses the target’s HTTP request message.

❝

The request result is then returned.

❞

5. URI Format

• URI (Uniform Resource Identifier) Unified Resource Identifier
• URL (Uniform Resource Locator, 统一资源定位符)

Reference Links

1. HTTP/3 From Start to Finish: Core Concepts

https://www.smashingmagazine.com/2021/08/http3-core-concepts-part1/