-
What is a bridge?
-
Creating a bridge
-
Connecting the bridge and veth devices
-
Assigning an IP to the bridge
-
Adding a physical network card to the bridge
-
Does the bridge need to be configured with an IP?
-
Common scenarios for bridges
-
Virtual Machines
-
Docker
What is a bridge?
First, a bridge is a virtual network device, so it has the characteristics of a network device, such as being able to configure an IP or MAC address; secondly, a bridge acts as a virtual switch, functioning similarly to a physical switch.
For ordinary network devices, there are only two ends: data coming in from one end will exit from the other end. For example, data received by a physical network card from the external network is forwarded to the kernel protocol stack, while data coming from the protocol stack is forwarded to the external physical network.
However, a bridge is different; it has multiple ports, and data can come in from any port. The outgoing port is determined based on the MAC address, similar to the principle of a physical switch.
Creating a bridge
Let’s first create a bridge:
dev@debian:~$ sudo ip link add name br0 type bridge
dev@debian:~$ sudo ip link set br0 up
When a bridge is first created, it is an independent network device with only one port connected to the protocol stack; the other ports are not connected to anything. This bridge has no actual functionality, as shown in the figure below:
Assuming eth0 is our physical network card, with an IP address of 192.168.3.21 and a gateway of 192.168.3.1.
Connecting the bridge and veth devices
Create a pair of veth devices and assign them IPs:
dev@debian:~$ sudo ip link add veth0 type veth peer name veth1
dev@debian:~$ sudo ip addr add 192.168.3.101/24 dev veth0
dev@debian:~$ sudo ip addr add 192.168.3.102/24 dev veth1
dev@debian:~$ sudo ip link set veth0 up
dev@debian:~$ sudo ip link set veth1 up
Connect veth0 to br0:
dev@debian:~$ sudo ip link set dev veth0 master br0
# Use the bridge link command to see which devices are connected to br0
dev@debian:~$ sudo bridge link
6: veth0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 2
At this point, the network looks like this:
After connecting br0 and veth0, several changes occur:
-
br0 and veth0 are interconnected, forming a bidirectional channel. -
The protocol stack and veth0 become a single channel; the protocol stack can send data to veth0, but data received by veth0 from the outside will not be forwarded to the protocol stack. -
The MAC address of br0 becomes the MAC address of veth0.
This means that the bridge intercepts data that veth0 would originally forward to the protocol stack, forwarding all data to the bridge instead, while the bridge can also send data to veth0.
Let’s verify this:
Pinging veth1 from veth0 fails:
dev@debian:~$ ping -c 1 -I veth0 192.168.3.102
PING 192.168.2.1 (192.168.2.1) from 192.168.2.11 veth0: 56(84) bytes of data.
From 192.168.2.11 icmp_seq=1 Destination Host Unreachable
--- 192.168.2.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
Why does veth0 fail to ping veth2 after joining the bridge? Let’s capture some packets:
# Since veth0's ARP cache does not have veth1's MAC address, it sends an ARP request first
# From the capture on veth1, we see that veth1 received the ARP request and replied
dev@debian:~$ sudo tcpdump -n -i veth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:43:48.353509 ARP, Request who-has 192.168.3.102 tell 192.168.3.101, length 28
21:43:48.353518 ARP, Reply 192.168.3.102 is-at 26:58:a2:57:37:e9, length 28
# From the capture on veth0, the packet was sent out and received a reply
dev@debian:~$ sudo tcpdump -n -i veth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:44:09.775392 ARP, Request who-has 192.168.3.102 tell 192.168.3.101, length 28
21:44:09.775400 ARP, Reply 192.168.3.102 is-at 26:58:a2:57:37:e9, length 28
# Checking the packet on br0, we find only the reply
dev@debian:~$ sudo tcpdump -n -i br0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:45:48.225459 ARP, Reply 192.168.3.102 is-at 26:58:a2:57:37:e9, length 28
From the above captures, we can see that both the request and reply processes are fine. The issue is that after veth0 receives the reply, it does not pass it to the protocol stack but gives it to br0. As a result, the protocol stack does not receive the MAC address of veth1, leading to communication failure.
Assigning an IP to the bridge
From the analysis above, it is clear that assigning an IP to veth0 is meaningless because even if the protocol stack sends packets to veth0, the reply packets will not come back. Therefore, we will assign the IP of veth0 to the bridge instead.
dev@debian:~$ sudo ip addr del 192.168.3.101/24 dev veth0
dev@debian:~$ sudo ip addr add 192.168.3.101/24 dev br0
Now the network looks like this:
Actually, there is still a connection between veth0 and the protocol stack, but since veth0 does not have an IP configured, the protocol stack will not send packets to veth0 when routing. Even if we force packets to go through veth0, veth0 will only send the data it receives from the other end to br0, so the protocol stack still cannot receive the corresponding ARP reply packet, resulting in communication failure. To make it clearer, we have removed the connection between the protocol stack and veth0; veth0 is like a network cable.
Now let’s ping veth1 via br0, and the result is successful:
dev@debian:~$ ping -c 1 -I br0 192.168.3.102
PING 192.168.3.102 (192.168.3.102) from 192.168.3.101 br0: 56(84) bytes of data.
64 bytes from 192.168.3.102: icmp_seq=1 ttl=64 time=0.121 ms
--- 192.168.3.102 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.121/0.121/0.121/0.000 ms
However, pinging the gateway still fails because this bridge only has two network devices: 192.168.3.101 and 192.168.3.102, and br0 does not know where 192.168.3.1 is.
dev@debian:~$ ping -c 1 -I br0 192.168.3.1
PING 192.168.3.1 (192.168.3.1) from 192.168.3.101 br0: 56(84) bytes of data.
From 192.168.3.101 icmp_seq=1 Destination Host Unreachable
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
Adding a physical network card to the bridge
Add eth0 to br0:
dev@debian:~$ sudo ip link set dev eth0 master br0
dev@debian:~$ sudo bridge link
2: eth0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4
6: veth0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 2
br0 does not differentiate between physical and virtual devices; to it, they are all network devices. Therefore, when eth0 joins br0, it ends up with the same fate as veth0: data packets received from the external network will be unconditionally forwarded to br0, effectively turning it into a network cable.
At this point, pinging the gateway through eth0 fails, but since br0 is connected to the external physical switch via eth0, all devices connected to br0 can ping the gateway. Here, the devices connected are veth1 and br0 itself, where veth1 is connected via veth0, while br0 can be understood as having its own built-in network card.
# Ping the gateway through eth0 fails
dev@debian:~$ ping -c 1 -I eth0 192.168.3.1
PING 192.168.3.1 (192.168.3.1) from 192.168.3.21 eth0: 56(84) bytes of data.
From 192.168.3.21 icmp_seq=1 Destination Host Unreachable
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
# Ping the gateway successfully through br0
dev@debian:~$ ping -c 1 -I br0 192.168.3.1
PING 192.168.3.1 (192.168.3.1) from 192.168.3.101 br0: 56(84) bytes of data.
64 bytes from 192.168.3.1: icmp_seq=1 ttl=64 time=27.5 ms
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 27.518/27.518/27.518/0.000 ms
# Ping the gateway successfully through veth1
dev@debian:~$ ping -c 1 -I veth1 192.168.3.1
PING 192.168.3.1 (192.168.3.1) from 192.168.3.102 veth1: 56(84) bytes of data.
64 bytes from 192.168.3.1: icmp_seq=1 ttl=64 time=68.8 ms
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 68.806/68.806/68.806/0.000 ms
Since eth0 has effectively become a cable, configuring an IP on eth0 no longer makes sense and may affect the routing choices of the protocol stack. For example, if we do not specify a network card during the ping operation, the protocol stack may prioritize eth0, leading to a ping failure. Therefore, we need to remove the IP from eth0.
# On my testing machine, since eth0 has an IP,
# when accessing the 192.168.3.0/24 network segment, eth0 is prioritized
dev@debian:~$ sudo route -v
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.3.1 0.0.0.0 UG 0 0 0 eth0
link-local * 255.255.0.0 U 1000 0 0 eth0
192.168.3.0 * 255.255.255.0 U 0 0 0 eth0
192.168.3.0 * 255.255.255.0 U 0 0 0 veth1
192.168.3.0 * 255.255.255.0 U 0 0 0 br0
# Since eth0 has joined br0, all packets it receives will be forwarded to br0,
# hence the protocol stack does not receive the ARP reply packet, causing ping to fail
dev@debian:~$ ping -c 1 192.168.3.1
PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
From 192.168.3.21 icmp_seq=1 Destination Host Unreachable
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
# Remove the IP from eth0
dev@debian:~$ sudo ip addr del 192.168.3.21/24 dev eth0
# Ping again, success
dev@debian:~$ ping -c 1 192.168.3.1
PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
64 bytes from 192.168.3.1: icmp_seq=1 ttl=64 time=3.91 ms
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.916/3.916/3.916/0.000 ms
# This is because after eth0 was removed, the routing table no longer includes it, so packets will go out via veth1
dev@debian:~$ sudo route -v
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.3.0 * 255.255.255.0 U 0 0 0 veth1
192.168.3.0 * 255.255.255.0 U 0 0 0 br0
# From here we can also see that since the original default route was through eth0, when eth0's IP was deleted, the default route disappeared. To connect to networks outside of 192.168.3.0/24, we need to manually add the default gateway back.
# Add default gateway, then ping the external network successfully
dev@debian:~$ sudo ip route add default via 192.168.3.1
dev@debian:~$ ping -c 1 baidu.com
PING baidu.com (111.13.101.208) 56(84) bytes of data.
64 bytes from 111.13.101.208: icmp_seq=1 ttl=51 time=30.6 ms
--- baidu.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 30.690/30.690/30.690/0.000 ms
After the above series of operations, the network looks like this:
Does the bridge need to be configured with an IP?
In our common physical switches, there are those that can be configured with an IP and those that cannot. Switches that cannot be configured with an IP typically connect via a COM port for configuration (simpler switches may not support any configuration). Switches that can be configured with an IP can be remotely connected for configuration after the IP is set, making it more convenient.
A bridge belongs to the latter type of switch; it has a built-in virtual network card that can be configured with an IP. One end of this virtual network card connects to the bridge, and the other end connects to the protocol stack. Like a physical switch, the operation of the bridge does not depend on this virtual network card, but the bridge’s operation does not mean the machine can connect to the internet; that depends on the networking method.
Remove the IP from br0:
dev@debian:~$ sudo ip addr del 192.168.3.101/24 dev br0
Now the network looks like this: one port of br0 connects to the switch via eth0, and the other port connects to veth0 and veth1:
Pinging the gateway is successful, indicating that in this case, not configuring an IP on br0 does not affect communication; packets can still go out from veth1:
dev@debian:~$ ping -c 1 192.168.3.1
PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.
64 bytes from 192.168.3.1: icmp_seq=1 ttl=64 time=1.24 ms
--- 192.168.3.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.242/1.242/1.242/0.000 ms
If there are no veth0 and veth1, deleting the IP from br0 would cause the network to become unreachable since there would be no device completely connected to the protocol stack.
Common scenarios for bridges
The examples above demonstrate the functionality of a bridge, but the deployment method shown in the example has little practical use; it is simpler to configure multiple IP addresses on a single network card. Here are two common deployment methods.
Virtual Machines
Virtual machines connect their internal network cards to br0 via tun/tap or other similar virtual network devices, achieving the same effect as a real switch. Packets sent out by the virtual machine first reach br0, which then forwards them to eth0 for transmission, without needing to go through the host machine’s protocol stack, thus improving efficiency.
Docker
Since containers run in their own separate network namespaces, they each have their own protocol stack. The situation is similar to that of virtual machines, but they use a different method to communicate with the outside world:
In the container, the gateway is configured as .9.1, and packets sent out first reach br0, then go to the host machine’s protocol stack. Since the destination IP is an external IP, and the host machine has IP forwarding enabled, the packets will be sent out via eth0. Since .9.1 is a private IP, NAT conversion is usually performed before sending out (both NAT conversion and IP forwarding need to be configured by the user). Because packets must go through the host machine’s protocol stack and undergo NAT conversion, the performance is not as good as the previous virtual machine method. The advantage is that containers are in a private network, which provides relatively higher security. (Since packets are uniformly forwarded from the IP layer through eth0, there is no issue with MAC addresses, and it works well in wireless network environments.)
In the two deployment methods above, each network card in the same subnet has its own protocol stack, so there is no issue with multiple ARPs as mentioned above.