Some production systems you face might make use of bonded network connections that you need to bridge in order to get VMs onto them. That bond may or may not have a native VLAN (in which case you bridge the bond), or it might have VLANs on top (in which case you want to bridge the VLANs), or perhaps you need to do both.
Let’s walk through an example where we have a bond that has a native VLAN, that also has the tagged VLAN 123
on top (and maybe a second VLAN 456
), all of which need to be separately bridged. This means we will have the bond (bond0
) with a matching bridge (br-bond0
), plus a VLAN on the bond (bond0.123
) with its matching bridge (br-vlan123
). It should look something like this.
+------+ +---------+ +---------------+
| eth0 |---| | +------------+ | Network one |
+------+ | |----------| br-bond0 |---| (native VLAN) |
| bond0 | +------------+ +---------------+
+------+ | |
| eth1 |---| |
+------+ +---------+ +---------------+
| | +---------+ +------------+ | Network two |
| +---| vlan123 |---| br-vlan123 |---| (tagged VLAN) |
| +---------+ +------------+ +---------------+
|
| +---------+ +------------+ +---------------+
+-----| vlan456 |---| br-vlan456 |---| Network three |
+---------+ +------------+ | (tagged VLAN) |
+---------------+
To make it more complicated, let’s say that the native VLAN on the bond needs a static IP and to operate at an MTU of 1500
while the other uses DHCP and needs MTU of 9000
.
OK, so how do we do that?
Start by creating the bridge, then later we create the interface that attaches to that bridge. When creating VLANs, they are created on the bond, but then attached as a slave to the bridge.
Create the bridge for the bond
First, let’s create the bridge for our bond. We’ll export some variables to make scripting easier, including the name, value for spanning tree protocol (SPT
) and MTU. Note that in this example the bridge will have an MTU of 1500
(but the bond itself will be 9000
to support other VLANs at that MTU size.)
BRIDGE=br-bond0
BRIDGE_STP=yes
BRIDGE_MTU=1500
OK so let’s create the bridge for the native VLAN on the bond (which doesn’t exist yet).
nmcli con add ifname "${BRIDGE}" type bridge con-name "${BRIDGE}"
nmcli con modify "${BRIDGE}" bridge.stp "${BRIDGE_STP}"
nmcli con modify "${BRIDGE}" 802-3-ethernet.mtu "${BRIDGE_MTU}"
By default this will look for an address with DHCP. If you don’t want that you can either set it manually:
nmcli con modify "${BRIDGE}" ipv4.method static ipv4.address 192.168.0.123/24 ipv6.method ignore
Or disable IP addressing:
nmcli con modify "${BRIDGE}" ipv4.method disabled ipv6.method ignore
Finally, bring up the bridge. Yes, we don’t have anything attached to it yet, but that’s OK.
nmcli con up "${BRIDGE}"
You should be able to see it with nmcli
and brctl
tools (if available on your distro), although note that there is no device attached to this bridge yet.
nmcli con
brctl show
Next, we create the bond to attach to the bridge.
Create the bond and attach to the bridge
Let’s create the bond. In my example I’m using active-backup (mode 1
) but your bond may use balance-rr
(round robin, mode 0
) or, depending on your switching, perhaps something like link aggregation control protocol (LACP) which is 802.3ad
(mode 4
).
Let’s say that your bond (we’re going to call bond0
) has two interfaces, which are eth0
and eth1
respectively. Note that in this example, although the native interface on this bond wants an MTU of 1500
, the VLANs which sit on top of the bond need a higher MTU of 9000
. Thus, we set the bridge to 1500
in the previous step, but we need to set the bond and its interfaces to 9000
. Let’s export those now to make scripting easier.
BOND=bond0
BOND_SLAVE0=eth0
BOND_SLAVE1=eth1
BOND_MODE=active-backup
BOND_MTU=9000
Now we can go ahead and create the bond, setting the options and the slave devices.
nmcli con add type bond ifname "${BOND}" con-name "${BOND}"
nmcli con modify "${BOND}" bond.options mode="${BOND_MODE}"
nmcli con modify "${BOND}" 802-3-ethernet.mtu "${BOND_MTU}"
nmcli con add type ethernet con-name "${BOND}-slave-${BOND_SLAVE0}" ifname "${BOND_SLAVE0}" master "${BOND}"
nmcli con add type ethernet con-name "${BOND}-slave-${BOND_SLAVE1}" ifname "${BOND_SLAVE1}" master "${BOND}"
nmcli con modify "${BOND}-slave-${BOND_SLAVE0}" 802-3-ethernet.mtu "${BOND_MTU}"
nmcli con modify "${BOND}-slave-${BOND_SLAVE1}" 802-3-ethernet.mtu "${BOND_MTU}"
OK at this point you have a bond specified, great! But now we need to attach it to the bridge, which is what will make the bridge actually work.
nmcli con modify "${BOND}" master "${BRIDGE}" slave-type bridge
Note that before we bring up the bond (or afterwards) we need to disable or delete any existing network connections for the individual interfaces. Check this with nmcli con
and delete or disable those connections. Note that this may disconnect you, so make sure you have a console to the machine.
Now, we can bring the bond up which will also activate our interfaces.
nmcli con up "${BOND}"
We can check that the bond came up OK.
cat /proc/net/bonding/bond0
And this bond should also now be on the network, via the bridge which has an IP set.
Now if you look at the bridge you can see there is an interface (bond0
) attached to it (your distro might not have brctl
).
nmcli con
ls /sys/class/net/br-bond0/brif/
brctl show
Bridging a VLAN on a bond
Now that we have our bond, we can create the bridged for our tagged VLANs (remember that the bridge connected to the bond is a native VLAN so it didn’t need a VLAN interface).
Create the bridge for the VLAN on the bond
Create the new bridge, which for our example is going to use VLAN 123
which will use MTU of 9000
.
VLAN=123
BOND=bond0
BRIDGE=br-vlan${VLAN}
BRIDGE_STP=yes
BRIDGE_MTU=9000
OK let’s go! (This is the same as the first bridge we created.)
nmcli con add ifname "${BRIDGE}" type bridge con-name "${BRIDGE}"
nmcli con modify "${BRIDGE}" bridge.stp "${BRIDGE_STP}"
nmcli con modify "${BRIDGE}" 802-3-ethernet.mtu "${BRIDGE_MTU}"
Again, this will look for an address with DHCP, so if you don’t want that, then disable it or set an address manually (as per first example). Then you can bring the device up.
nmcli con up "${BRIDGE}"
Create the VLAN on the bond and attach to bridge
OK, now we have the bridge, we create the VLAN on top of bond0
and then attach it to the bridge we just created.
nmcli con add type vlan con-name "${BOND}.${VLAN}" ifname "${BOND}.${VLAN}" dev "${BOND}" id "${VLAN}"
nmcli con modify "${BOND}.${VLAN}" master "${BRIDGE}" slave-type bridge
nmcli con modify "${BOND}.${VLAN}" 802-3-ethernet.mtu "${BRIDGE_MTU}"
If you look at bridges now, you should see the one you just created, attached to a VLAN device (note, your distro might not have brctl
).
nmcli con
brctl show
And that’s about it! Now you can attach VMs to those bridges and have them on those networks. Repeat the process for any other VLANs you need on to of the bond.
9 thoughts on “How to create bridges on bonds (with and without VLANs) using NetworkManager”
Thank youu for the great content <3 <3
Great, you solved my problem and you are the only resource that I have found with my google search skills.
I only needed the hint how to add a created bond to an already existing bridge and nmtui wasn’t able to do that.
Thank you very much !
Great, I’m glad it was helpful!
Excellent! Saved my day.
I have Ubuntu 18.04 with three physical 1G Ethernet interfaces already slaved under bond0 as pure L2 without any L3 (no IP addresses).
bond0 is untagged native with an IP interface. This is the network that needs to be bridged to a LXD container.
I can use brctl or iproute2 to crease the bridge.
I want the bridge to only be L2, just as the bonded eth.
It seems like I just create the bridge, attach to bond0 and, after creating the container, attach to the contaise.
Hopefully
Thank you very much for this. Very helpful, clear, concise approach.
can you explain please where you sould tag the the interfaces and for what reason because i dont understand of the concept
When do you choose whether to go with a native VLAN or create VLANs on the bond that you bridge?
I guess it depends if you need more than one network to work on the machine and also how the connected switches are configured.