Creating HTTP and HTTPS Servers and Clients with Node.js

Creating HTTP and HTTPS Servers and Clients with Node.js

Source | https://blog.csdn.net/ligang2585116/article/details/72827781

The HyperText Transfer Protocol (HTTP) is the most widely used network protocol on the Internet. All WWW files must comply with this standard. HTTP was originally designed to provide a method for publishing and receiving HTML pages. It belongs to the “Application Layer” of the seven-layer network protocol as shown in the figure below.

Creating HTTP and HTTPS Servers and Clients with Node.js

HTTP Server

Creating an HTTP Server

Method 1: Callback Method

var server = http.createServer((request, response) => {    // Triggered when accepting client requests    ...});server.listen(10000, 'localhost', 511, () => {   // Start listening   ...});

Method 2: Event Listener Method

var server = http.createServer();// Triggered when accepting client requestsserver.on('request', (request, response) => {    ...});server.listen(10000, 'localhost', 511);// Start listeningserver.on('listening', () => {    ...});

Note:

  • The backlog parameter in server.listen(port, [host], [backlog], [callback]) is an integer that specifies the maximum number of client connections in the waiting queue. Once this length is exceeded, the HTTP server will start rejecting connections from new clients. The default value is 511.

  • When making an HTTP request to the server, two requests are sent. One is the request made by the user, and the other is automatically sent by the browser for the page’s bookmark icon (default is favicon.ico).

Closing the Server

server.close();// The close event will be triggered when the server is closedserver.on('close', () => {...});

Timeout

server.setTimeout(60 * 1000, () => {   console.log('Timeout occurred');});// Or through event formserver.setTimeout(60 * 1000);server.on('timeout', () => {...});

Note: The default timeout is 2 minutes.

Error

server.on('error', (e) => {    if(e.code === 'EADDRINUSE') {        // Port is occupied    }});

Getting Client Request Information

The data event is triggered when data is read from the client request stream, and the end event is triggered when all data from the client request stream has been read.

Creating HTTP and HTTPS Servers and Clients with Node.js

GET Request

server.on('request', (request, response) => {    if(request.url !== '/favicon.ico') {        /* GET request */        var params = url.parse(req.url, true).query;        // Or        // var params = querystring.parse(url.parse(request.url).query);        // Process according to parameters        // ...        // End request        response.end();    }  });

POST Request

server.on('request', (request, response) => {    request.setEncoding('utf-8');    if(request.url !== '/favicon.ico') {        let result = '';        request.on('data', (data) => {            result += data;        });        request.on('end', () => {            var params = JSON.parse(postData);            console.log(`Data received: ${result}`);        });        // End this request        response.end();    }    // End this request    response.end(JSON.stringify({status: 'success'}));});

Converting URL Strings and Query Strings

The querystring module: Converts query strings in the URL (after ? and before #).

querystring.stringify(obj, [sep], [eq])querystring.parse(str, [sep], [eq], [option])

sep: Separator, default &

eq: Assignment character, default =

options: {maxKeys: number} Specifies the number of properties in the converted objectlet str = querystring.stringify({name: 'ligang', age: 27});console.log(str); // name=ligang&age=27let obj = querystring.parse(str);console.log(obj); // { name: 'ligang', age: '27' }

The url module: Converts complete URL strings.

url.parse(urlStr, [parseQueryString])

parseQueryString: If true, will convert query strings to objects using querystring; default is false.

url.resolve(from, to);

Combines the two into one path, where from and to can be either relative or absolute paths.

// http://ligangblog.com/javascript/a?a=1url.resolve('http://ligangblog.com/javascript/', 'a?a=1'); // http://ligangblog.com/a?a=1url.resolve('http://ligangblog.com/javascript/', '/a?a=1');

Note: For specific merging rules, please refer to “Node.js Authority Guide” – 8.1 HTTP Server.

Creating HTTP and HTTPS Servers and Clients with Node.js

var urlStr = 'http://ligangblog.com/javascript/?name=lg&uid=1#a/b/c';console.log(url.parse(urlStr, true));/*Url {    protocol: 'http:',    slashes: true,    auth: null,    host: 'ligangblog.com',    port: null,    hostname: 'ligangblog.com',    hash: '#a/b/c',    search: '?name=lg&uid=1',    query: { name: 'lg', uid: '1' },    pathname: '/javascript/',    path: '/javascript/?name=lg&uid=1',    href: 'http://ligangblog.com/javascript/?name=lg&uid=1#a/b/c' }*/

Sending Server Response Stream

response.writeHead(statusCode, [reasonPhrase], [headers]);// Orresponse.setHeader(name, value);

Some common fields included in the response header:

Creating HTTP and HTTPS Servers and Clients with Node.js

Example:

response.writeHead(200, {'Content-Type': 'text/plain',                          'Access-Control-Allow-Origin': 'http://localhost'});// Orresponse.statusCode = 200;response.setHeader('Content-Type', 'text/plain');response.setHeader('Access-Control-Allow-Origin', 'http://localhost');

Difference between writeHead and setHeader:

writeHead: This method sends the response header when called.

setHeader: Sends the response header when the write method is called for the first time.

/* Get the value of a specific field in the response header */response.getHeader(name);/* Delete a response field value */response.removeHeader(name);/* This property indicates whether the response header has been sent */response.headersSent;/* Add a header information at the end of the response data */response.addTrailers(headers);

Example:

// Must add Trailer field in the response header, and its value must be set to the field name specified in the additional response headersresponse.write(200, {'Content-Type': 'text/plain', 'Trailer': 'Content-MD5'});response.write('....');response.addTrailers({'Content-MD5', '...'});response.end();

Special Note:

In a fast network with small data amounts, Node will send the data directly to the operating system’s kernel cache, and then retrieve the data from that cache to send to the requester; if the network is slow or the data amount is large, Node usually caches the data in memory and sends the memory data to the requester via the operating system’s kernel cache when the other party can accept the data.

If response.write returns true, it indicates that the data was written directly to the operating system’s kernel cache; if false, it indicates that it is temporarily cached in memory. Each time, response.end() must be called to end the response.

Creating HTTP and HTTPS Servers and Clients with Node.js

Response timeout will trigger the timeout event; if the connection is interrupted before the response.end() method is called, the close event will be triggered.

/* Set request timeout to 2 minutes */response.setTimeout(2 * 60 * 1000, () => {  console.error('Request timed out!'); });// Orresponse.setTimout(2 * 60 * 1000);response.on('timeout', () => {  console.error('Request timed out!');});/* Connection interruption */response.on('close', () => {  console.error('Connection interrupted!');});
/** * HTTP Server * Created by ligang on 17/5/28. */import http from 'http';var server = http.createServer();// Triggered when accepting client requestsserver.on('request', (request, response) => {    if(request.url !== '/favicon.ico') {        response.setTimeout(2 * 60 * 1000, () => {           console.error('Request timed out!');        });        response.on('close', () => {            console.error('Request interrupted!');        });        let result = '';        request.on('data', (data) => {            result += data;        });        request.on('end', () => {            console.log(`Server data received: ${result}`);            response.statusCode = 200;            response.write('Received!');            response.end(); // End this request        });    }});server.listen(10000, 'localhost', 511);// Start listeningserver.on('listening', () => {    console.log('Started listening');});server.on('error', (e) => {    if(e.code === 'EADDRINUSE') {        console.log('Port is occupied');    }else {        console.log(`An error occurred: ${e.code}`);    }});

HTTP Client

Node.js can easily send requests to any website and read the response data from the website.

var req = http.request(options, callback);// GET requestvar req = http.get(options, callback);// Send data to target websitereq.write(chunk, [encoding]);// End this requestreq.end([chunk], [encoding]);// Abort this requestreq.abort();

Among them, options is used to specify the target URL address. If this parameter is a string, it will automatically be converted to an object using the parse method in the url module. Note: The http.get() method can only be used to request data in GET mode, and there is no need to call req.end(), as Node.js will call it automatically.

/** * HTTP Client * Created by ligang on 17/5/30. */import http from 'http';const options = {        hostname: 'localhost',        port: 10000,        path: '/',        method: 'post'    },    req = http.request(options);req.write('Hello, server');req.end();req.on('response', (res) => {    console.log(`Status code: ${res.statusCode}`);    let result = '';    res.on('data', (data) => {        result += data;    });    res.on('end', () => {        console.log(`Client received response: ${result}`);    })});req.setTimeout(60* 1000, () => {    console.log('Timeout occurred');    req.abort();});req.on('error', (error) => {    if(error.code === 'ECONNRESET') {        console.log('Socket port timeout');    }else {        console.log(`Send error: ${error.code}`);    }});

Proxy Server

/** * HTTP Proxy * Created by ligang on 17/5/30. */import http from 'http';import url from 'url';/** * Server */const server = http.createServer(async (req, res) => {    // req.setEncoding('utf-8');    /* Timeout 2 minutes */    res.setTimeout(2 * 60 * 1000, () => {        // ...    });    /* Connection interruption */    res.on('close', () => {        // ...    });    let options = {},        result = "";    options = await new Promise((resolve, reject) => {        if(req.method === 'GET') {            options = url.parse('http://localhost:10000' + req.url);            resolve(options);        }else if(req.method === 'POST') {            req.on('data', (data) => {                result += data;            });            req.on('end', () => {                options = url.parse('http://localhost:10000' + req.url);                // POST request must specify                options.headers = {                    'content-type': 'application/json',                };                resolve(options);            });        }    });    options.method = req.method;    let content = await clientHttp(options, result ? JSON.parse(result) : result);    res.setHeader('Content-Type', 'text/html');    res.write('<html><head><meta charset="UTF-8" /></head>')    res.write(content);    res.write('</html>');    // End this request    res.end();});server.listen(10010, 'localhost', 511);/* Start listening */server.on('listening', () => {    // ...});/* Listen for errors */server.on('error', (e) => {    console.log(e.code);    // ...});/** * Client * @param options Request parameters * @param data Request data */async function clientHttp(options, data) {    let output = new Promise((resolve, reject) => {        let req = http.request(options, (res) => {            let result = '';            res.setEncoding('utf8');            res.on('data', function (chunk) {                result += chunk;            });            res.on('end', function () {                resolve(result);            });        });        req.setTimeout(60000, () => {            console.error(`Connection to backend timed out ${options.href}`);            reject();            req.abort();        });        req.on('error', err => {            console.error(`Error connecting to backend ${err}`);            if (err.code === 'ECONNRESET') {                console.error(`Socket timeout ${options.href}`);            } else {                console.error(`Error connecting to backend ${err}`);            }            reject();            req.abort();        });        // If there is request data, send the request data        if (data) {            req.write(JSON.stringify(data));        }        req.end();    });    return await output;}

Note:

POST requests must specify header information, otherwise a socket hang-up error will occur.

After obtaining options, you need to re-specify its method options.method = req.method;

HTTPS Server

  • HTTPS uses the https protocol, default port 443;

  • HTTPS requires applying for a certificate from a certificate authority;

  • The data transmitted between the HTTPS server and client is encrypted data.

Creating Public Key, Private Key, and Certificate

(1) Create a Private Key

openssl genrsa -out privatekey.pem 1024

(2) Create a Certificate Signing Request

openssl req -new -key privatekey.pem -out certrequest.csr

(3) Obtain a certificate, online certificates must be signed by a certificate authority; below is just to create a certificate for learning purposes.

openssl x509 -req -in certrequest.csr -signkey privatekey.pem -out certificate.pem

(4) Create a pfx file

openssl pkcs12 -export -in certificate.pem -inkey privatekey.pem -out certificate.pfx

HTTPS Service

Creating an HTTPS server is similar to creating an HTTP server, but it requires adding a certificate and setting options when creating the HTTPS server.

import https from 'https';import fs from 'fs';var pk = fs.readFileSync('privatekey.pem'),    pc = fs.readFileSync('certificate.pem');var opts = {    key: pk,    cert: pc};var server = https.createServer(opts);

The opts parameter is an object used to specify various options configured when creating the HTTPS server. Below are just a few necessary options:

Creating HTTP and HTTPS Servers and Clients with Node.js

HTTPS Client

const options = {        hostname: 'localhost',        port: 1443,        path: '/',        method: 'post',        key: fs.readFileSync('privatekey.pem'),        cert: fs.readFileSync('certificate.pem'),        rejectUnauthorized: false,        agent: false // Specify to pick a currently closed https.Agent from the connection pool    },    req = https.request(options);// Orconst options = {        hostname: 'localhost',        port: 1443,        path: '/',        method: 'post',        key: fs.readFileSync('privatekey.pem'),        cert: fs.readFileSync('certificate.pem'),        rejectUnauthorized: false,    };// Explicitly specify the https.Agent objectoptions.agent = new https.Agent(options);var req = https.request(options);
Note: In a normal HTTPS service, the server does not verify the client’s certificate (but the certificate must be provided), and a man-in-the-middle can successfully complete the TLS handshake with the client and server; However, if the man-in-the-middle does not have the certificate private key, they cannot impersonate the server and client to establish a TLS connection.
Of course, if you have the certificate private key, proxying the HTTPS website corresponding to the certificate is certainly not a problem, so here the private key and public key are just for format writing and have little significance, as long as the data returned from the request is handed back to the browser for parsing, the task is completed.

Creating HTTP and HTTPS Servers and Clients with Node.js

Creating HTTP and HTTPS Servers and Clients with Node.js

Related posts

  1. Unirest: A Lightweight HTTP Request Tool for Java
  2. What Is a Network Protocol? Understanding Network Processes
  3. Understanding the Fundamental Differences Between HTTP and HTTPS
  4. Building an AI Camera with Python, Raspberry Pi, and YOLO
  5. Why Use RPC When We Have HTTP?
  6. HTTP Persistent Connections and HttpClient Connection Pool
  7. Essential Knowledge for Beginners in Cybersecurity: Understanding HTTP Packets
  8. Learn Interface Testing from Scratch – HTTP Message Structure
  9. Efficient HTTP Large Data Requests Using Go Language Streaming Processing
  10. Understanding TCP/IP Protocols
  11. How to Set Up an IP Address
  12. Getting Started with HTTP Requests Using Httpie in Python
  13. Basics of HTTP Protocol
  14. Why Does HTTPS Return Large Data While HTTP Returns Line by Line?
  15. Detailed Explanation of HTTP Protocol
  16. A Brief Discussion on HTTP and RPC
  17. Comprehensive Node.js Resource Compilation with 40K Stars!
  18. Understanding the Evolution and Ecosystem of Arm Server Products
  19. Simple Configuration of Clash in Kali

Leave a Comment