This series of articles first studies the core concepts of P2P networks and then analyzes application examples in the libp2p-rust library in detail, laying a solid foundation for future development of P2P network applications.
P2P Network
P2P (Peer-to-Peer) is a network technology that allows different computers in a network to share various computing resources, such as CPU, network bandwidth, and storage. The most common applications of P2P technology are file sharing in networks and blockchain networks, which do not rely on a central server or intermediary to connect multiple clients; the user’s computer acts as both client and server.
Since P2P networks are distributed systems, they do not have a single point of failure like central servers, making them highly fault-tolerant.
Now, let’s take a look at the core concepts of P2P networks:
Transport Protocol
P2P networks generally use TCP/UDP transport layer protocols at the bottom layer. Due to the diversity of P2P node applications, various application layer protocols, such as HTTP, gRPC, and custom protocols, are used on top of TCP/UDP transport layer protocols. To effectively utilize resources, P2P networks listen and parse multiple protocols on a single connection, which is called multiplexing: multiple logical substreams can coexist on the same underlying (TCP) connection. You can check the yamux library for more details.
Node Identification
Nodes in a P2P network need a unique identifier so that other nodes can find them. Nodes in a P2P network establish secure communication with other nodes using a public and private key pair (asymmetric public key encryption). In a P2P network, the node identifier is called PeerId, which is obtained by encrypting the node’s public key with a hash.
Security Rules
The key pairs and node identity identifiers allow nodes to establish secure, authenticated communication channels. However, this is just one aspect of security; nodes also need to implement an authorization framework based on business logic that establishes rules for which nodes can perform which types of operations, etc.
Node Routing
A node in a P2P network first needs to find other nodes to communicate. This is achieved by maintaining a node routing table. However, in a P2P network, there are thousands of nodes that change dynamically (i.e., nodes joining and leaving), making it difficult for a single node to maintain a complete and accurate routing table for all nodes in the network. Therefore, the node routing table is typically maintained by a series of routing nodes.
Messages
Nodes in a P2P network can send messages to specific nodes or broadcast messages. Using a publish/subscribe model, nodes subscribe to topics of interest, and all nodes subscribed to that topic can receive and send messages. This technique is also commonly used to transmit message content across the entire network.
Stream Multiplexing
In a P2P network, multiple independent “logical” streams are allowed to share a common P2P transport layer. Stream multiplexing helps optimize the overhead of establishing network connections between nodes. Multiplexing is common in backend service development, where clients can establish an underlying network connection with the server and then reuse different streams over that underlying network connection.
libp2p
Writing a network layer for P2P applications from scratch is a massive undertaking. We will use the underlying p2p network library – libp2p, which makes it easier to build P2P applications. libp2p is a modular system that supports three programming languages: Rust, Go, and JS. Many popular projects use libp2p as the underlying P2P network, such as IPFS, Filecoin, Polkadot, and Substrate.
libp2p breaks down the basic concepts of P2P networks into different modules that can be combined for various application scenarios.
We will first familiarize ourselves with the components of libp2p and how to use libp2p to develop peer-to-peer networks through a simple Ping program.
PING
This example is very simple; it mainly involves one node sending a ping message to another node and then waiting for the other node to return a pong message.
Create a project called: libp2p-learn
master:p2p Justin$ cargo new libp2p-learn Created binary (application) `libp2p-learn` packagemaster:p2p Justin$ cd libp2p-learn/master:libp2p-learn Justin$ code .Add libp2p and tokio dependencies to the Cargo.toml file:
[dependencies]libp2p = "0.46"tokio = { version = "1.19", features = ["full"] }Then create a ping.rs file in the src/bin/ directory:
use std::error::Error;
use libp2p::{ futures::StreamExt, identity, ping::{Ping, PingConfig}, swarm::SwarmEvent, Multiaddr, PeerId, Swarm,};
#[tokio::main]async fn main() -> Result<(), Box<dyn Error>> { // Generate key pair let key_pair = identity::Keypair::generate_ed25519();
// Generate node unique identifier peerId based on the public key of the key pair let peer_id = PeerId::from(key_pair.public()); println!("Node ID: {peer_id}");
// Declare Ping network behavior let behaviour = Ping::new(PingConfig::new().with_keep_alive(true));
// Transport let transport = libp2p::development_transport(key_pair).await?;
// Network management module let mut swarm = Swarm::new(transport, behaviour, peer_id);
// Randomly open a port to listen on the node swarm.listen_on("/ip4/0.0.0.0/tcp/0".parse()?)?;
// Get the remote node address from command line arguments and connect. if let Some(remote_peer) = std::env::args().nth(1) { let remote_peer_multiaddr: Multiaddr = remote_peer.parse()?; swarm.dial(remote_peer_multiaddr)?; println!("Connecting to remote node: {remote_peer}"); }
loop { // Match network events match swarm.select_next_some().await { // Listen events SwarmEvent::NewListenAddr { address, .. } => { println!("Local listening address: {address}"); } // Network behavior events SwarmEvent::Behaviour(event) => println!("{:?}", event), _ => {} } }}-
Network Behavior: The transport defines how to send byte streams over the network, while network behavior defines what type of byte streams to send; here we send ping/pong messages.
-
Network Management Module Swarm: Used to manage all active and pending connections between nodes and manage the state of all opened substreams. Swarm is created through transport, network behavior, and node PeerId.
-
Node Address: /ip4/0.0.0.0/tcp/0 indicates that a random TCP port is opened for listening on all IP addresses of the local machine.
Open a terminal and run:
cargo run --bin pingmaster:libp2p-learn Justin$ cargo run --bin ping Compiling libp2p-learn v0.1.0 (/Users/Justin/workspace_rust_exercise/network-study/p2p/libp2p-learn) Finished dev [unoptimized + debuginfo] target(s) in 8.65s Running `target/debug/ping`Node ID: 12D3KooWR7H9SwB2yiFBKvzcVGFdpeKmuFG9qDTBTvuuuDarASSTLocal listening address: /ip4/127.0.0.1/tcp/58645You can see that the PeerId and listening address have been printed out.
Open another terminal and run:
cargo run --bin ping /ip4/127.0.0.1/tcp/58645master:libp2p-learn Justin$ cargo run --bin ping /ip4/127.0.0.1/tcp/58645 Finished dev [unoptimized + debuginfo] target(s) in 0.36s Running `target/debug/ping /ip4/127.0.0.1/tcp/58645`Node ID: 12D3KooWCUFTHNMJrR1p8vkFEFFYm4J8iPA1Wh6x2Dya5qmU1xdLConnecting to remote node: /ip4/127.0.0.1/tcp/58645Local listening address: /ip4/127.0.0.1/tcp/58727Event { peer: PeerId("12D3KooWR7H9SwB2yiFBKvzcVGFdpeKmuFG9qDTBTvuuuDarASST"), result: Ok(Pong) }Event { peer: PeerId("12D3KooWR7H9SwB2yiFBKvzcVGFdpeKmuFG9qDTBTvuuuDarASST"), result: Ok(Ping { rtt: 1.234008ms }) }We can see that the connection to the previous node /ip4/127.0.0.1/tcp/58645 was successful, and we also received the sent ping/pong messages.
In the next article, we will analyze the P2P chat program in detail.