Sets of virtual machines are connected to a virtual bridges (e.g.
virbr1) and as they are isolated, can use the same subnet range and set of IPs. However, NATing becomes a problem because the host won’t know which VM to return the traffic to.
To solve this problem, we can use network namespaces and some veth (virtual Ethernet) devices for each private network we want to NAT. The veth devices come as an interconnected pair of interfaces (think patch cable) and we will use two pairs for each network namespace; one which patches into the provider network (e.g.
192.168.0.0/24) and another which patches to the virtual machine’s private network (e.g.
By providing each private network with is own unique upstream routable IP and applying NAT rules inside each namespace separately we can avoiding any conflict.
Create a provider bridge
You’ll need a bridge to a physical network, which will act as your upstream route (like a “provider” network).
ip link add name br0 type bridge ip link set br0 up ip link set eth0 up ip link set eth0 master br0
We create our namespace to patch in the veth devices and hold the router and isolated NAT rules. As this is for the purpose of NATing multiple private networks, I’m making it sequential and calling this
nat1 (for our first one, then I’ll call the next one
ip netns add nat1
First veth pair
Our first veth peer interfaces pair will be used to connect the namespace to the upstream bridge (
br0). Give them a name that makes sense to you; here I’m making it sequential again and specifying the purpose. Thus,
peer1-br0 will connect to the upstream
peer1-gw1 will be our routable IP in the namespae.
ip link add peer1-br0 type veth peer name peer1-gw1
Adding the veth to provider bridge
Now we need to add the
peer1-br0 interface to the upstream provider bridge and bring it up. Note that we do not set an IP on this, it’s a patch lead. The IP will be on the other end in the namespace.
brctl addif br0 peer1-br0 ip link set peer1-br0 up
First gateway interface in namespace
Next we want to add the
peer1-gw1 device to the namespace, give it an IP on the routable network, set the default gateway and bring the device up. Note that if you use DHCP you can do that, here I’m just setting an IP statically to
192.168.0.100 and gateway of
ip link set peer1-gw1 netns nat1 ip netns exec nat1 ip addr add 192.168.0.100/24 dev peer1-gw1 ip netns exec nat1 ip link set peer1-gw1 up ip netns exec nat1 ip route add default via 192.168.0.1
Second veth pair
Now we create the second veth pair to connect the namespace into the private network. For this example we’ll be connecting to
virbr0 network, where our first set of VMs are running. Again, give them useful names.
ip link add peer1-virbr0 type veth peer name peer1-gw2
Adding the veth to private bridge
Now we need to add the
peer1-virbr0 interface to the
virbr0 private network bridge. Note that we do not set an IP on this, it’s a patch lead. The IP will be on the other end in the namespace.
brctl addif virbr0 peer1-virbr0 ip link set peer1-virbr0 up
Second gateway interface in namespace
Next we want to add the
peer1-gw2 device to the namespace, give it an IP on the private network and bring the device up. I’m going to set this to the default gateway of the VMs in the private network, which is
ip link set peer1-gw2 netns nat1 ip netns exec nat1 ip addr add 10.0.0.1/24 dev peer1-gw2 ip netns exec nat1 ip link set up dev peer1-gw2
Enable NAT in the namespae
So now we have our namespace with patches into each bridge and IPs on each network. The final step is to enable network address translation.
ip netns exec nat1 iptables -t nat -A POSTROUTING -o peer1-gw1 -j MASQUERADE ip netns exec nat1 iptables -A FORWARD -i peer1-gw1 -o peer1-gw2 -m state --state RELATED,ESTABLISHED -j ACCEPT ip netns exec nat1 iptables -A FORWARD -i peer1-gw2 -o peer1-gw1 -j ACCEPT
You can see the rules with standard iptables in the netspace.
ip netns exec nat1 iptables -t nat -L -n
OK so logging onto the VMs, they should a local IP (e.g.
10.0.0.100, a default route to
10.0.0.1 and have upstream DNS set. Test that they can ping the gateway, test they can ping the DNS and test that they can ping a DNS name on the Internet.
Rinse and repeat
This can be applied for other virtual machine networks as required. There is no-longer any need for the VMs there to have unique IPs, they can overlap eachother.
What you do need to do is create a new network namespace, create two new sets of veth pairs (with a useful name) and pick another IP on the routable network. The virtual machine gateway IP will be the same in each namespace, that is