-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Alexis Bauvin says:
====================
net: Add VRF support for VXLAN underlay
v6 -> v7:
- proper locking for device in udp_tunnel following Sabrina Dubroca's advice
v5 -> v6:
- remove automatic rebinding patch following Roopa Prabhu's advice
v4 -> v5:
- move test script to its own patch (6/6)
- add schematic for test script
- apply David Ahern comments to the test script
v3 -> v4:
- rename vxlan_is_in_l3mdev_chain to netdev_is_upper master
- move it to net/core/dev.c
- make it return bool instead of int
- check if remote_ifindex is zero before resolving the l3mdev
- add testing script
v2 -> v3:
- fix build when CONFIG_NET_IPV6 is off
- fix build "unused l3mdev_master_upper_ifindex_by_index" build error with some
configs
v1 -> v2:
- move vxlan_get_l3mdev from vxlan driver to l3mdev driver as
l3mdev_master_upper_ifindex_by_index
- vxlan: rename variables named l3mdev_ifindex to ifindex
v0 -> v1:
- fix typos
We are trying to isolate the VXLAN traffic from different VMs with VRF as shown
in the schemas below:
+-------------------------+ +----------------------------+
| +----------+ | | +------------+ |
| | | | | | | |
| | tap-red | | | | tap-blue | |
| | | | | | | |
| +----+-----+ | | +-----+------+ |
| | | | | |
| | | | | |
| +----+---+ | | +----+----+ |
| | | | | | | |
| | br-red | | | | br-blue | |
| | | | | | | |
| +----+---+ | | +----+----+ |
| | | | | |
| | | | | |
| | | | | |
| +----+--------+ | | +--------------+ |
| | | | | | | |
| | vxlan-red | | | | vxlan-blue | |
| | | | | | | |
| +------+------+ | | +-------+------+ |
| | | | | |
| | VRF | | | VRF |
| | red | | | blue |
+-------------------------+ +----------------------------+
| |
| |
+---------------------------------------------------------+
| | | |
| | | |
| | +--------------+ | |
| | | | | |
| +---------+ eth0.2030 +---------+ |
| | 10.0.0.1/24 | |
| +-----+--------+ VRF |
| | green|
+---------------------------------------------------------+
|
|
+----+---+
| |
| eth0 |
| |
+--------+
iproute2 commands to reproduce the setup:
ip link add green type vrf table 1
ip link set green up
ip link add eth0.2030 link eth0 type vlan id 2030
ip link set eth0.2030 master green
ip addr add 10.0.0.1/24 dev eth0.2030
ip link set eth0.2030 up
ip link add blue type vrf table 2
ip link set blue up
ip link add br-blue type bridge
ip link set br-blue master blue
ip link set br-blue up
ip link add vxlan-blue type vxlan id 2 local 10.0.0.1 dev eth0.2030 \
port 4789
ip link set vxlan-blue master br-blue
ip link set vxlan-blue up
ip link set tap-blue master br-blue
ip link set tap-blue up
ip link add red type vrf table 3
ip link set red up
ip link add br-red type bridge
ip link set br-red master red
ip link set br-red up
ip link add vxlan-red type vxlan id 3 local 10.0.0.1 dev eth0.2030 \
port 4789
ip link set vxlan-red master br-red
ip link set vxlan-red up
ip link set tap-red master br-red
ip link set tap-red up
We faced some issue in the datapath, here are the details:
* Egress traffic:
The vxlan packets are sent directly to the default VRF because it's where the
socket is bound, therefore the traffic has a default route via eth0. the
workaround is to force this traffic to VRF green with ip rules.
* Ingress traffic:
When receiving the traffic on eth0.2030 the vxlan socket is unreachable from
VRF green. The workaround is to enable *udp_l3mdev_accept* sysctl, but
this breaks isolation between overlay and underlay: packets sent from
blue or red by e.g. a guest VM will be accepted by the socket, allowing
injection of VXLAN packets from the overlay.
This patch series fixes the issues describe above by allowing VXLAN socket to be
bound to a specific VRF device therefore looking up in the correct table.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>- Loading branch information
Showing
8 changed files
with
228 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| #!/bin/bash | ||
| # SPDX-License-Identifier: GPL-2.0 | ||
|
|
||
| # This test is for checking VXLAN underlay in a non-default VRF. | ||
| # | ||
| # It simulates two hypervisors running a VM each using four network namespaces: | ||
| # two for the HVs, two for the VMs. | ||
| # A small VXLAN tunnel is made between the two hypervisors to have the two vms | ||
| # in the same virtual L2: | ||
| # | ||
| # +-------------------+ +-------------------+ | ||
| # | | | | | ||
| # | vm-1 netns | | vm-2 netns | | ||
| # | | | | | ||
| # | +-------------+ | | +-------------+ | | ||
| # | | veth-hv | | | | veth-hv | | | ||
| # | | 10.0.0.1/24 | | | | 10.0.0.2/24 | | | ||
| # | +-------------+ | | +-------------+ | | ||
| # | . | | . | | ||
| # +-------------------+ +-------------------+ | ||
| # . . | ||
| # . . | ||
| # . . | ||
| # +-----------------------------------+ +------------------------------------+ | ||
| # | . | | . | | ||
| # | +----------+ | | +----------+ | | ||
| # | | veth-tap | | | | veth-tap | | | ||
| # | +----+-----+ | | +----+-----+ | | ||
| # | | | | | | | ||
| # | +--+--+ +--------------+ | | +--------------+ +--+--+ | | ||
| # | | br0 | | vrf-underlay | | | | vrf-underlay | | br0 | | | ||
| # | +--+--+ +-------+------+ | | +------+-------+ +--+--+ | | ||
| # | | | | | | | | | ||
| # | +---+----+ +-------+-------+ | | +-------+-------+ +---+----+ | | ||
| # | | vxlan0 |....| veth0 |.|...|.| veth0 |....| vxlan0 | | | ||
| # | +--------+ | 172.16.0.1/24 | | | | 172.16.0.2/24 | +--------+ | | ||
| # | +---------------+ | | +---------------+ | | ||
| # | | | | | ||
| # | hv-1 netns | | hv-2 netns | | ||
| # | | | | | ||
| # +-----------------------------------+ +------------------------------------+ | ||
| # | ||
| # This tests both the connectivity between vm-1 and vm-2, and that the underlay | ||
| # can be moved in and out of the vrf by unsetting and setting veth0's master. | ||
|
|
||
| set -e | ||
|
|
||
| cleanup() { | ||
| ip link del veth-hv-1 2>/dev/null || true | ||
| ip link del veth-tap 2>/dev/null || true | ||
|
|
||
| for ns in hv-1 hv-2 vm-1 vm-2; do | ||
| ip netns del $ns || true | ||
| done | ||
| } | ||
|
|
||
| # Clean start | ||
| cleanup &> /dev/null | ||
|
|
||
| [[ $1 == "clean" ]] && exit 0 | ||
|
|
||
| trap cleanup EXIT | ||
|
|
||
| # Setup "Hypervisors" simulated with netns | ||
| ip link add veth-hv-1 type veth peer name veth-hv-2 | ||
| setup-hv-networking() { | ||
| hv=$1 | ||
|
|
||
| ip netns add hv-$hv | ||
| ip link set veth-hv-$hv netns hv-$hv | ||
| ip -netns hv-$hv link set veth-hv-$hv name veth0 | ||
|
|
||
| ip -netns hv-$hv link add vrf-underlay type vrf table 1 | ||
| ip -netns hv-$hv link set vrf-underlay up | ||
| ip -netns hv-$hv addr add 172.16.0.$hv/24 dev veth0 | ||
| ip -netns hv-$hv link set veth0 up | ||
|
|
||
| ip -netns hv-$hv link add br0 type bridge | ||
| ip -netns hv-$hv link set br0 up | ||
|
|
||
| ip -netns hv-$hv link add vxlan0 type vxlan id 10 local 172.16.0.$hv dev veth0 dstport 4789 | ||
| ip -netns hv-$hv link set vxlan0 master br0 | ||
| ip -netns hv-$hv link set vxlan0 up | ||
| } | ||
| setup-hv-networking 1 | ||
| setup-hv-networking 2 | ||
|
|
||
| # Check connectivity between HVs by pinging hv-2 from hv-1 | ||
| echo -n "Checking HV connectivity " | ||
| ip netns exec hv-1 ping -c 1 -W 1 172.16.0.2 &> /dev/null || (echo "[FAIL]"; false) | ||
| echo "[ OK ]" | ||
|
|
||
| # Setups a "VM" simulated by a netns an a veth pair | ||
| setup-vm() { | ||
| id=$1 | ||
|
|
||
| ip netns add vm-$id | ||
| ip link add veth-tap type veth peer name veth-hv | ||
|
|
||
| ip link set veth-tap netns hv-$id | ||
| ip -netns hv-$id link set veth-tap master br0 | ||
| ip -netns hv-$id link set veth-tap up | ||
|
|
||
| ip link set veth-hv netns vm-$id | ||
| ip -netns vm-$id addr add 10.0.0.$id/24 dev veth-hv | ||
| ip -netns vm-$id link set veth-hv up | ||
| } | ||
| setup-vm 1 | ||
| setup-vm 2 | ||
|
|
||
| # Setup VTEP routes to make ARP work | ||
| bridge -netns hv-1 fdb add 00:00:00:00:00:00 dev vxlan0 dst 172.16.0.2 self permanent | ||
| bridge -netns hv-2 fdb add 00:00:00:00:00:00 dev vxlan0 dst 172.16.0.1 self permanent | ||
|
|
||
| echo -n "Check VM connectivity through VXLAN (underlay in the default VRF) " | ||
| ip netns exec vm-1 ping -c 1 -W 1 10.0.0.2 &> /dev/null || (echo "[FAIL]"; false) | ||
| echo "[ OK ]" | ||
|
|
||
| # Move the underlay to a non-default VRF | ||
| ip -netns hv-1 link set veth0 vrf vrf-underlay | ||
| ip -netns hv-1 link set veth0 down | ||
| ip -netns hv-1 link set veth0 up | ||
| ip -netns hv-2 link set veth0 vrf vrf-underlay | ||
| ip -netns hv-2 link set veth0 down | ||
| ip -netns hv-2 link set veth0 up | ||
|
|
||
| echo -n "Check VM connectivity through VXLAN (underlay in a VRF) " | ||
| ip netns exec vm-1 ping -c 1 -W 1 10.0.0.2 &> /dev/null || (echo "[FAIL]"; false) | ||
| echo "[ OK ]" |