Understanding the Relationship Between TCP, Sockets, and HTTP

Recently, I needed to implement an HTTP interface for a project. So, I took this opportunity to explain the relationship between HTTP and sockets, as well as their connection to TCP.

First, it’s essential to understand that in the network layer architecture, the HTTP protocol belongs to the application layer, while the TCP protocol belongs to the transport layer. This means they are both protocols, which are rules established by the communicating parties; without these rules, two hosts cannot communicate.

Based on what we’ve learned, two hosts must establish a common protocol at the transport layer to communicate. Whether a connection needs to be established at the transport layer depends on the protocol; TCP requires a connection, while UDP does not. The differences between TCP and UDP are beyond the scope of this article, so we will not discuss them here. Since most data transmission today uses the TCP protocol, mastering TCP is very important.

Understanding the Relationship Between TCP, Sockets, and HTTP

When using the TCP protocol to send data at the transport layer, the first step is to complete the TCP three-way handshake process. Why is this process necessary? It ensures the accuracy of data transmission, meaning that data can be reliably transferred to another host. There is plenty of information available online about how to complete the three-way handshake process, so feel free to look it up.

The three-way handshake to establish a connection is more of a theoretical process. In other words, I can tell you about the three-way handshake, but you need to help me implement this process. The actual implementation relies on sockets. A socket is a comprehensive set of functions provided by the operating system for establishing connections, sending data, and disconnecting. It serves as an interface for external use. Note that I mentioned the operating system, which means that the socket interface functions encapsulated by different operating systems may vary. In Linux, the most commonly used socket functions are socket(), bind(), listen(), accept(), connect(), and close(), while there are slight differences in Windows.

At this point, I hope everyone understands that TCP is merely a protocol at the transport layer, a set of rules established by the communicating parties, while a socket is the specific implementation of this protocol. If you are skilled enough, you can create your own transport layer protocol for two communicating hosts and write code to implement this process. However, generally, no one would do this. Why? Because the workload would be immense. This challenge is not just about defining the protocol and writing the code; it also involves adapting both hosts to this protocol. The server side is manageable since it is under your control, typically running on Linux. However, the client side is a different story, as it involves potentially thousands of machines with different operating systems. You would either need to adapt your protocol to each system or negotiate with operating system vendors to implement your protocol. Therefore, only influential global companies and organizations can accomplish such tasks; it is usually impractical and unnecessary for the average person.

Having discussed all this, I want to emphasize that for two parties to communicate, they must first establish a connection using the TCP protocol at the transport layer. Once the connection is established, they can start sending data. This leads to the following questions:

1. If I want to send data with different structures and rules, how do I do this?

2. When I send data, I will definitely receive a response. How do I handle the returned data?

If the above two questions are not clear to you, don’t worry. Keep reading, and you might understand better.

Based on the two questions above, it is necessary to establish a set of rules for data transmission and reception at the application layer. HTTP is a classic protocol at the application layer; it is the most widely used network transmission protocol on the Internet, and all WWW files must comply with this standard. Of course, the application layer includes more than just HTTP; there are also classic protocols like Telnet, FTP, SMTP, etc. The communicating parties must send and receive data according to the specified formats of these protocols. Depending on their data transmission needs, they can also create their own application layer protocols to meet localized requirements. As long as there is a need, you can freely add application layer protocols.

So why is it so difficult to add transport layer protocols, while adding application layer protocols is relatively easy?

In simple terms: Transport layer protocols operate at the operating system level, while application layer protocols operate at the application software level.

Thus, adding a transport layer protocol is a massive undertaking because it requires updates at the operating system level. In contrast, adding an application layer protocol is simpler since it only involves adding it to the client and server of the software or app you are developing.

To illustrate with a relatable example, if you want to travel from point A to point B, you need to build a road. The design plans for this road are analogous to TCP, while the construction process is akin to sockets. You cannot build blindly; you must follow the design plans, just as sockets must adhere to the theoretical framework of TCP. Building a road is a costly endeavor, making it impractical to add transport layer protocols casually. Once the road is built, you can travel in various ways—walking, running, cycling, driving, etc.—as long as you enjoy the journey. The method of travel represents the application layer protocols, with HTTP being one of those methods.

In summary, TCP is a protocol at the transport layer, while a socket is the encapsulation of this protocol, providing an interface for applications to call. HTTP is a protocol at the application layer, serving as a data encapsulation method. When initiating an HTTP request, the underlying transport layer must establish a connection between the two machines, which involves completing the TCP three-way handshake.

Below, I provide an example of an HTTP encapsulation, which only includes the client side, written during a project. You can use it as a reference.

//http interface
int http_post_openapi(const char *pIP, const char *pServ, int port, const char *pSendValue, char *pRecvValue){
    int sockfd, ret, i, h;
    struct sockaddr_in servaddr;
    char szHttpHead[1024], buf[8192], szSendValueLength[128], szSendBuffer[4096];
    int iLen = 0;
    fd_set t_set1;
    struct timeval tv;
    if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) < 0 ) {
        printf("Failed to create network connection, this thread will terminate---socket error!\n");
        exit(0);
    };
    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family = AF_INET;
    servaddr.sin_port = htons(port);
    if (inet_pton(AF_INET, pIP, &servaddr.sin_addr) <= 0 ) {
        printf("Failed to create network connection, this thread will terminate--inet_pton error!\n");
        exit(0);
    };
    if (connect(sockfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) < 0){
        printf("Failed to connect to server, connect error!\n");
        exit(0);
    }
    printf("Connected to the remote host\n");
    if (pSendValue == NULL || pRecvValue == NULL) {
        close(sockfd);
        printf("\nEither pSendValue or pRecvValue is NULL!!!\n");
        return -1;
    }
    char szSendConver[4096] = {0};
    TrimAll(pSendValue, szSendConver);
    iLen = strlen(szSendConver);
    memset(szSendValueLength, 0, 128);
    sprintf(szSendValueLength, "%d", iLen);
    printf("szSendValueLength after sprintf is :%s\n", szSendValueLength);
    memset(szHttpHead, 0, 256);
    strcat(szHttpHead, "POST ");
    strcat(szHttpHead, pServ);
    strcat(szHttpHead, " HTTP/1.1\r\n");
    strcat(szHttpHead, "Host: ");
    strcat(szHttpHead, pIP);
    strcat(szHttpHead, ":");
    char cPort[6];
    sprintf(cPort,"%ld",port);
    strcat(szHttpHead, cPort);
    strcat(szHttpHead, "\r\n");
    strcat(szHttpHead, "User-Agent: Apache-HttpClient/4.1.1\r\n");
    strcat(szHttpHead, "Accept: */*\r\n");
    strcat(szHttpHead, "Content-Length: ");
    strcat(szHttpHead, szSendValueLength);
    strcat(szHttpHead, "\r\n");
    strcat(szHttpHead, "Content-Type: application/json; charset=UTF-8");
    printf("szSendValueLength is :%s\n", szSendValueLength);
    strcat(szHttpHead, "\r\n\r\n");
    memset(szSendBuffer, 0, 4096);
    strcat(szSendBuffer, szHttpHead);
    strcat(szSendBuffer, szSendConver);
    strcat(szSendBuffer, "\r\n\r\n");
    printf("Print SendBuffer before write :\n%s\n",szSendBuffer);
    ret = write(sockfd,szSendBuffer,strlen(szSendBuffer));
    if (ret < 0) {
        printf("Send failed! Error code is %d, error message is '%s'\n",errno, strerror(errno));
        exit(0);
    }else{
        printf("Message sent successfully, a total of %d bytes were sent!\n\n", ret);
    }
    FD_ZERO(&t_set1);
    FD_SET(sockfd, &t_set1);
    memset(buf, 0, sizeof(buf));
    i= read(sockfd, buf, sizeof(buf)-1);
    if (i==0) {
        close(sockfd);
        printf("Detected that the remote closed the connection while reading the data packet, this thread will terminate!\n");
        return -1;
    }
    close(sockfd);
    // Here, find the HTTP RESPONSE result code. If it is 200, then it is successful, extract the body and assign it to pRecvBuff; otherwise, return -1.
    char *pRet = NULL;
    char *pStart = NULL;
    char *pEnd = NULL;
    //pRet = strstr(buf, "HTTP/1.1 200 OK");
    pRet = strstr(buf, "HTTP/1.1 200");
    if(!pRet) {
        cout << "HTTP Response Error!!!" << endl;
        return -1;
    }
    string sRecv;
    utf82gb(buf, sRecv);
    printf("sRecv is :\n%s\n", sRecv.c_str());
    memset(buf, 0, sizeof(buf));
    strncpy(buf, sRecv.c_str(), sRecv.length());
    pStart = strstr(buf, "{");
    printf("pStart address is :0x%x\n", pStart);
    pEnd = strrchr(buf, '}') + 1;
    printf("pEnd address is :0x%x\n", pStart);
    // if(pStart != NULL && pEnd != NULL) {
    strncpy(pRecvValue, pStart, (int)(pEnd - pStart));
    //userlog("Message returned successfully - message, request message:\n%s\n ******** Return message:\n%s\n", szSendBuffer, buf);
    //} else {
    printf("\nbuf中未找到匹配的数据!!!\n");
    userlog("Message return failed - message, request message:\n%s\n ******** Return message:\n%s\n", szSendBuffer, buf);
    return -2;
    //}
    return 0;
}

For more exciting content, please follow: A Bit of Moonlight

Related posts

Leave a Comment Cancel reply