Unlike bridged veth, the created veth is not linked with any bridge interface. Instead, the veth is routed on a network layer using configured routes. The advantage of routed veth over the bridged veth is that the interfaces are isolated. The containers are not connected on the link layer.
Routed veth requires manual setup of the network environment. You either need software for dynamic routing, such as OSPF or BGP, or to configure static routes for the IP addresses routed to containers. Without any setup, the routed addresses will be reachable only locally between the host and containers.
Let's see how can the routed veth be used:
Create new routed interface in container
osctl ct netif new routed myct01 eth0
Add a single IP address:
osctl ct netif ip add myct01 eth0 126.96.36.199/32
Route a larger network and assign one IP address from the network:
osctl ct netif ip add --route-as 10.0.0.0/24 myct01 eth0 10.0.0.1/24
It's important to distinguish addresses that are routed to the container and addresses that are assigned to the container's interfaces. The assigned addresses are a subset of the routed addresses. It is possible to route larger networks and assign selected addresses to the container's interface.
Routes are managed using
osctl ct netif route commands:
osctl ct netif route add myct01 eth0 10.5.5.0/24
The command above will route network
10.5.5.0/24 to the container, but no
address will be assigned to its interface yet. Addresses are managed using
osctl ct netif ip commands:
osctl ct netif ip add myct01 eth0 10.5.5.1/24
To make the usage more straightforward,
osctl ct netif ip add will
automatically add route for the added address, unless there is one already
present. This behaviour can be controlled by CLI options, see man osctl for
Before you start using it, it's important to understand how the routed veth works. Routing addresses from the host to the container is straightforward, routes are added to the host's veth interface, e.g.:
ip route add <routed address> dev $hostveth
However, routing from the container is more complicated. The container's default route has to be routed via the host's IP address. For this purpose, osctld creates a dummy interface on the host and adds one IPv4 address through which all IPv4 traffic is routed. IPv6 traffic is routed through link-local addresses assigned to veth interfaces on the host.
# Host ip link add osrtr0 type dummy ip address add 255.255.255.254/32 dev osrtr0 # Containers ip route add 255.255.255.254/32 dev $ctveth ip route add default via 255.255.255.254 dev $ctveth
188.8.131.52/32 would be routed to container from the host like this:
ip route add 184.108.40.206/32 dev $hostveth
We've added route for
220.127.116.11/32 through the container's veth interface
on the host.
In the container, we'd first add the routed IP address to the interface and then set the default route via the host's address:
ip address add 18.104.22.168/32 dev eth0 ip route add 255.255.255.254/32 dev eth0 ip route add default via 255.255.255.254 dev eth0
You don't actually have to do any of that manually, because osctld manages routes and addresses on its own. The example configuration would be created using osctl as:
osctl ct netif new routed myct01 eth0 osctl ct netif ip add myct01 eth0 22.214.171.124/32