Bridging domains to tagged VLANs in Xen
5 Comments Published by tristanb February 27th, 2007 in Development, ProjectsThe base Xen installation is configured such that all domains share a bridge to the first available ethernet interface. Although this makes testing VMs quick and painless, for an upcoming project I want to be able to isolate individual DomUs from the rest of the network. Specifically, I want the Dom0 to tag all traffic from a given DomU onto a VLAN. Here’s a simple example:
Untagged traffic is sent to eth0 and ends up in Dom0. VLAN 2 traffic is set to eth0.2, which is bridged with DomU 1′s interface. The same occurs with VLAN 3 and DomU 2. One key point in this setup is that the DomUs are completely unaware of the fact that they are utilizing a VLAN, all the work is done within the bridges in Dom0. The Dom0 is also unaware of the traffic within the VLAN, as it does not have an active address on the eth0.X or vlanbrX interfaces.
To have Xen automatically create these bridges, I wrote the network-bridge-vlan and network-multi-vlan scripts. They have been tested wtih Xen 3 running on Debian Sarge, but should work with most distributions. Make both scripts executable and place them in /etc/xen/scripts. The VLAN configuratin is done within the network-multi-vlan script. To activate the VLANs, edit /etc/xen/xend-config.sxp to use the configuration line
(network-script network-multi-vlan)
To use a new VLAN bridge for an interface within your DomU, change the xen configuration file for the DomU so that the network interface has a bridge specified:
vif = [ 'bridge=vlanbr2' ]
Within the DomU, there are two things that may need to be configured. First, if your ethernet card does not natively support VLAN tags, you will have to set the maximum MTU to 1496 to make room for the tag. The command
ifconfig eth0 mtu 1496
accomplishes this in linux. Another problem is related to network optimizations done within Xen. With the DomUs bridged to VLAN interfaces, these optimizations need to be disabled or tcp and udp connections will fail. This is done by disabling transmit checksum offloading:
ethtool -K eth0 tx off
Both should be done in a script at startup. For Debian, I created the script /etc/network/if-up.d/xen containing:
#!/bin/sh
case "$IFACE" in
eth0)
ethtool -K $IFACE tx off
ifconfig $IFACE mtu 1496
;;
esac
5 Responses to “Bridging domains to tagged VLANs in Xen”
- 1 Pingback on Jul 19th, 2007 at 4:04 am
Nice article, congrats!!
I was wondering if you ever tried to make something similar, or if you know that this works:
The DOMU’s are firewalls with multiple VLAN interfaces on the inside, and one normal interface on the outside, like this:
DOMu1 — eth0 (external-if) — xenbr0 — eth0 (Dom0)
— eth2.2 (internal-if)
— eth2.3 (internal-if)
— eth2.4 (internal-if)
DOMu2 — eth0 (external-if) — xenbr0 — eth0 (Dom0)
— eth2.2 (internal-if)
— eth2.3 (internal-if)
— eth2.4 (internal-if)
I’ve tried a series of bridges/setups that never worked when trying to ping DOMu2 eth2.2 from DOMu1 (thats why i didnt included any bridge information on the eth2.x interfaces of the DOMu’s). Nothing that i’ve tried so far made those DOMu’s vlan interfaces to communicate.
Any toughts about that would b extremely valuable,
Thanks a lot and keep up the good work!
Hello very good !!! i needed this thanks, !
Hey. Thanks for that script. It appears to work happilly when netdev=bond0 too.
Hi,
Nice scripts, very useful!
I’d just like to add that disabling Spanning Tree (STP) is not the wisest thing to do by default.
I suggest enabling STP, and remarking the line “brctl setfd ${bridge} 0″ en replace “off” by “on” on line “brctl stp ${bridge} off” Routing loops are nasty.
from ‘man brctl’:
brctl stp controls this bridge instance’s participaâ€
tion in the spanning tree protocol. If is “on” or “yes” the
STP will be turned on, otherwise it will be turned off. When turned
off, the bridge will not send or receive BPDUs, and will thus not
participate in the spanning tree protocol. If your bridge isn’t the
only bridge on the LAN, or if there are loops in the LAN’s topology,
DO NOT turn this option off. If you turn this option off, please know
what you are doing.
Patch:
— network-bridge-vlan.old 2007-07-26 13:32:40.145832732 -0400
network-bridge-vlan 2007-07-26 13:19:50.593479432 -0400
@@ -76,8 76,8 @@
# Don’t create the bridge if it already exists.
if ! brctl show | grep -q ${bridge} ; then
brctl addbr ${bridge}
- brctl stp ${bridge} off
- brctl setfd ${bridge} 0
brctl stp ${bridge} on
#brctl setfd ${bridge} 0
fi
ip link set ${bridge} up
}
Cheers,
Otto