When I graduated, the country still had a system of job allocation. My best friend Xiaoming was assigned to a big city called Database, where he could sit in a high-end data center every day, specializing in SQL query optimization. His job was stable and comfortable.
In the neighboring dorm, Xiaobai was sent to Compiler Town, where he specifically compiled C source files into EXE programs. Although it was tiring, the technical content was very high, with good pay and plenty of holidays.
I was not a good student, a typical underachiever, having to retake the CET-4 exam twice before passing. I was assigned to a village whose name I didn’t even know, supposedly to handle some HTTP requests. This village was actually just an old computer, but I was comforted by the fact that I could go online and occasionally communicate with my friends.
However, the counselor said that we all had bright futures.
1
Http Server 1.0
HTTP was a novel concept that sparked a bit of interest in my work, preventing me from sinking into despair.
As soon as I started work, the operating system chief threw a pile of documents at me: “This is the HTTP protocol, read it in two days!”
With my level of English, I couldn’t finish reading these dozens of pages of the English HTTP protocol even if I didn’t eat or sleep for two days. I decided to take it slow.
Two weeks later, I finally had a rough understanding of what HTTP was: it was simply a matter of browsers on some computers sending a predefined text (HTTP request) to my old computer, which would then process it (usually retrieving a file with an HTML extension from the hard drive) and send that file back in text form (HTTP response). It was that simple.
The only tricky part was that I had to ask the operating system to establish a TCP connection channel beneath the HTTP layer, as all text data had to be received and sent through these TCP channels, which were established using sockets.
Once I understood the principles, I quickly developed the first version of the program, which looked like this:
(Note from the coder: For details, refer to the article “Zhang Dapeng’s Socket”)
Look, these sockets, bind, listen, accept… are all interfaces provided by the operating system. All I could do was assemble them: first listen on port 80, then enter an infinite loop. If a connection request came in, I would accept it, create a new socket, and finally use that socket to receive and send HTTP data.
The chief named my program Http Server, version 1.0.
This name sounded quite advanced, and I liked it.
I excitedly brought it for testing. The program started, sitting on port 80, and after a while, there was a connection request. I quickly accepted it, established a new socket, and succeeded! Next, I needed to read the HTTP request from the socket.
But the receive call was so slow; I waited a full 100 milliseconds without a response! I was blocked!
The operating system chief said: “Don’t worry, I’m also waiting to read data from the network card. Once it’s done, I’ll copy it to you.”
I was happy to have some free time to rest.
But the operating system chief said: “Don’t relax; there are many browsers trying to connect. You can’t just sit here!”
I asked, what should I do if I can’t relax? The receive call is blocking on your end; what else can I do but join the blocking queue and yield the CPU to let others use it?
The chief said: “Sigh, didn’t you learn about multi-processing in college? You’re clearly single-threaded; once you block, it’s over. Find a way to use multi-processing, with each process handling one request!”
The chief was right; I had forgotten about concurrent programming with multiple processes.
2
Http 2.0: Multi-Processing
The idea of multi-processing is very simple. After accepting a connection, for this new socket, I wouldn’t handle it in the main process but would create a new child process to take over. This way, the main process wouldn’t be blocked on receive and could continue accepting new connections.
I rewrote the code and upgraded the Http server to V2.0. This time, it ran much smoother and could handle many connections concurrently.
At that time, the web was just emerging, and not many people were accessing my Http Server. There were only a few dozen connection requests per minute, which I could easily handle.
Since it was a new thing, I still had the capital to boast to Xiaoming, who was working with databases, and Xiaobai, who was working with compilers, telling them I was a network expert.
In a few years, the web rapidly developed, and my old machine couldn’t keep up anymore. I replaced it with a powerful server and moved to a data center with a pleasant climate.
Now there were hundreds of connection requests every second, and some connections lasted quite a long time. I often had to create hundreds or thousands of processes to handle them, each process consuming a lot of system resources. It was clear that the operating system chief was overwhelmed.
He said: “We can’t keep doing this; so many processes are exhausting me just with process switching.”
“How about I use threads instead of processes for each socket connection?”
“That might be better, but I still have to switch threads. Think of a way to limit the number!”
I wondered how to limit it. I could only say that at any given time, I could only support x connections, and other connections would have to queue up and wait.
This was definitely not a good solution.
3
Http Server 3.0: Select Model
The chief said: “Let’s think carefully. For me, each socket connection is a so-called file descriptor (File Descriptor, abbreviated as fd, which is an integer). Behind this fd is a simple data structure, but we are using a very heavyweight thing—processes—to represent read and write operations on it, which is a bit wasteful.“
I said: “How about we switch back to a single-process model? But then we would be back to square one, and a single blocking receive would halt everything.”
“Single-process is not impossible, but we need to change our working method.”
“Change to what?” I couldn’t figure out what the chief was hinting at.
“Think about the essence of your blocking; it’s still because the browser hasn’t sent the data yet. Naturally, I can’t give it to you, and since you’re eager to read, I have to block you. In a single-process situation, once you block, you can’t do anything else.”
“Right, that’s it.”
“So after you accept a client connection, you can’t be so eager to read. Let’s do this: each of your socket fds has a number. You tell me these numbers, and you can block and rest.”
I asked: “Isn’t this the same as before? Originally, I blocked on the receive call, and now I’m still blocking.”
“Let me finish. I will check these numbered sockets in the background. If I find that these sockets can read or write, I will mark the corresponding socket and wake you up to process the data. After you’re done, you tell me your socket fds again and go back to blocking, repeating this cycle.”
I began to understand: “This is a way for us to communicate. I tell you what I’m waiting for, then block. If an event occurs, you wake me up to do the work.”
“Exactly. The key point is that you wait for my notification. After I wake you from the blocking state, you must traverse all the socket fds to see who has a mark and handle those with marks. I call this method select.”
I rewrote the Http server using the select method, abandoning the model of one process per socket request. Now, one process could handle all sockets.
4
Http Server 4.0: epoll
This select method ran for a while and worked quite well. I just had to tell the chief the socket fds and wait for his notification.
One day, I casually asked the chief: “How many socket fds can I tell you at most?”
“1024.”
“So that means I can monitor at most 1024 sockets with one process?”
“Yes, you might consider using more processes.”
This was one way, but after using the select method for a while, I discovered its drawbacks. The biggest problem was that after recovering from blocking, I had to traverse over 1000 socket fds to see if there were any flags to handle.
In reality, many sockets are not active; for a period, the browser may not send any data. Out of those 1000+ sockets, only a few dozen may need to be processed, but I have to check all the socket fds, which is quite tedious.
Couldn’t the chief just tell me which sockets had changed?
I shared this idea with the chief, and he said: “Hmm, as the traffic increases, the select method is no longer sufficient. We need to keep up with the times. I’ve thought of a new method called epoll.”
“You see, using epoll is actually similar to select,” the chief continued, “The difference lies in steps 3 and 4. I will only tell you which sockets can read or write. You only need to handle those ‘ready’ sockets.”
“It seems the chief has thought this through. This method is much simpler for me.”
I upgraded the Http Server again using epoll. Since I no longer needed to traverse the entire set, only handling the sockets that had changes, the system’s processing capability saw a significant leap.
My Http Server became widely popular, with countless users worldwide. Eventually, my friend Xiaoming from the database field found out and asked me: “Everyone says you can easily support tens of thousands of concurrent connections. Is that true?”
I humbly replied: “Too much praise; actually, I still need to optimize the system.”
He said: “Impressive, you got lucky!”
I responded: “Didn’t the counselor say back then that everyone has a bright future?”
(The End)
Related historical articles recommended by the coder:
Java EE
I am a thread
I am a Java class
Java: The Birth of an Empire
The Birth of JDBC
The Aftermath of JDBC
An Unruly JDBC Driver
JSP: The Decline of an Assembler
Javascript: A Loser’s Comeback
Spring Essence Series (1) – Dependency Injection
Spring Essence Series (2) – AOP
Http Adventure (Part 1)
Http Adventure (Part 2) – The Secrets of Struts
Three-Tier Architecture and MVC
Java Empire: Java Bean (Part 1)
Java Empire: Java Bean (Part 2)
Java Empire: Functional Programming (Part 1)
Java Empire: Functional Programming (Part 2)
Computer Networks
I am a router
I am a network card
The Ming Courier of TCP/IP
The Ming Cabinet of TCP/IP
The Jiliao Commander of TCP/IP
Zhang Dapeng’s Socket
Why did IE injure Chrome and Firefox?
The Second Interview with Browser Village
The Self-Narrative of the Thrifty IE
The Birth of EMail
The Birth of EMail (Part 2)
Operating Systems
I am a process
CPU Forrest Gump
CPU Forrest Gump’s Troubles
I am a keyboard
I am a hard disk (Part 1)
I am a hard disk (Part 2)
Those Annoying Synchronization and Mutual Exclusion Issues
The Birth of the Von Neumann Computer
Databases
Xiao Li’s Database Journey (Part 1)
Xiao Li’s Database Journey (Part 2)
Zhang Dapeng Studies Databases
The Wealthy and Xiao Wang of Database Village
You have only seen the tip of the iceberg; for more exciting articles, check out the “Coder’s Comeback” WeChat public account. Reply with “m” or “directory” to see more articles.
Do you have insights to share with everyone? Contributions are welcome! My contact information: WeChat: liuxinlehan QQ: 3340792577
Public Account: Coder’s Comeback
The “Coder’s Comeback” public account was created by a former IBM architect with 15 years of experience, sharing lessons learned in programming and the workplace.