org-notes/equinix/design/network-namespace-testing.org

Linux Networking For Fun and Profit

• Setting up "hosts" with network namespaces
  • Basic network namespace set up
  • Point to point connectivity
  • Bridging across virtual ethernet interfaces
  • Complete the set up with loopback interfaces
  • Cleaning up

Setting up "hosts" with network namespaces

Network namespaces give us a neat way to simulate networked machines. This walks through using Linux network namespaces to configure a set of "hosts" in the following configuration.

          Host 3
         /    \
      veth1b  veth2b
     /             \
   veth1a          veth2a
   Host 1         Host 2

• Host 1 - 192.168.65.1
• Host 2 - 192.168.65.2
• Host 3 - 192.168.65.3

In this configuration, even though these are all on the same subnet Host 1 only has a connection to Host 3 so can't directly reach Host 2.

Basic network namespace set up

All these steps are performed on Fedora 39 (Linux 6.7.5), but this should be possible on any modern Linux distro.

First we'll create all the network namespaces

sudo ip netns add host1
sudo ip netns add host2
sudo ip netns add host3

Then we'll create all the (virtual) interfaces we need. These paired virtual ethernets act as direct connections between the host machines.

sudo ip link add veth1a type veth peer name veth1b
sudo ip link add veth2a type veth peer name veth2b

So far we've only created the interfaces, we haven't assigned them to our network namespaces, so let's do that now:

sudo ip link set veth1a netns host1
sudo ip link set veth1b netns host3

sudo ip link set veth2a netns host2
sudo ip link set veth2b netns host3

Point to point connectivity

At this point we've got the hosts mostly configured, each host has the correct interfaces, but we have to bring them up and assign IPs. Let's start with assigning ips to just Host 1 and Host 2 to prove we can't communicate just yet.

sudo ip netns exec host1 ip addr add 192.168.65.1/24 dev veth1a
sudo ip netns exec host1 ip link set veth1a up

sudo ip netns exec host2 ip addr add 192.168.65.2/24 dev veth2a
sudo ip netns exec host2 ip link set veth2a up

sudo ip netns exec host1 ping -c1 192.168.65.2
# this should fail with 100% packet loss

We know there's a path from Host 1 to Host 2 through Host 3 though, but before we do that, let's just make sure we can communicate point-to-point from Host 1 to Host 3.

We'll do this by adding an IP to veth1b and bringing it up, and then pinging that IP from Host 1.

sudo ip netns exec host3 ip addr add 192.168.65.3/24 dev veth1b
sudo ip netns exec host3 ip link set veth1b up
sudo ip netns exec host3 ip link set veth2b up

sudo ip netns exec host1 ping -c1 192.168.65.3

Host 1 to Host 3 succeeds because our veth pair is connected directly.

Bridging across virtual ethernet interfaces

So that's easy, we can communicate point-to-point, but we still can't figure out how to get to Host 2 from Host 1. We can even check the ARP table to see why.

sudo ip netns exec host1 arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.65.3             ether   1a:60:c6:d9:2b:a0   C                     veth1a
192.168.65.2                     (incomplete)                              veth1a

ARP isn't able to figure out what mac address is for the owner of 192.168.65.2. We have an veth pair on Host 3 connected to Host1 and another veth pair connected to Host 2, but we can't get from Host 1 to Host 2.

We can solve this at layer 2 by creating a bridge interface that just sends packets along from one interface to the other.

First let's remove the IP we put on the veth1b.

sudo ip netns exec host3 ip addr del 192.168.65.3/24 dev veth1b

Now let's create that bridge interface, so we can allow the networking stack to pass packets from veth1b to veth2b.

sudo ip netns exec host3 ip link add br0 type bridge
sudo ip netns exec host3 ip link set veth1b master br0
sudo ip netns exec host3 ip link set veth2b master br0

And now, instead of assigning the IPs to the underlying interfaces, we'll just assign an IP to the bridge interface and bring it up

sudo ip netns exec host3 ip addr add 192.168.65.3/24 dev br0
sudo ip netns exec host3 ip link set up br0

Let's test our configuration from Host 3, we should now be able to reach both Host 1 and Host 2 by leveraging our underlying veth interfaces.

sudo ip netns exec host3 ping -c1 192.168.65.1
sudo ip netns exec host3 ping -c1 192.168.65.2

And now, let's try from Host 1 to Host 2, and back.

sudo ip netns exec host1 ping -c1 192.168.65.2
sudo ip netns exec host2 ping -c1 192.168.65.1

Finally, we can see that our pings reach, and if we look at the ARP table you can confirm that the addresses match the address of the veth device on Host 2.

sudo ip netns exec host1 arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.65.2             ether   46:ca:27:82:5a:c3   C                     veth1a
192.168.65.3             ether   b6:66:d9:d0:4d:39   C                     veth1a

Complete the set up with loopback interfaces

There's still something funny though, if a host tries to reach itself, it can't yet, and that's because we never brought up the loopback interface.

sudo ip netns exec host2 ip a show lo
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

We'll bring it up for all 3 hosts and we're in business.

  for i in 1 2 3
  do
      echo "============:: Host ${i}"
      sudo ip netns exec host${i} ip link set lo up

      for j in 1 2 3
      do
	  sudo ip netns exec host${i} ping -c1 192.168.65.${j}
      done
      echo "======================"
  done

Cleaning up

All the resources should be in the network namespaces, so we should be able to easily clean up by removing the namespaces.

  for i in 1 2 3
  do
      sudo ip netns delete host${i}
  done