Docker Swarms, vxlan kernel module

Snapcase · March 16, 2019 at 9:01 PM

Hi,

I'm using Libreelec 9.0.1 with the latest docker add-on provided from the repo. I have other docker nodes that aren't running Libreelec. Although I can add libreleec to my swarm just fine, any containers using an overlay network (either created manually or via the manager) fail to start since it seems to require vxlan be compiled in the kernel.

I've tried multiple containers and methods and overlay networks always fail, regardless of whether its in a swarm or not.

eg:

network sandbox join failed: subnet sandbox join failed for "10.0.2.0/24": error creating vxlan interface: operation not supported"

Creating a vxlan manually as root throws the same error, even if docker isn't in the mix at all.

tl;dr: Docker overlay networks used in docker swarms require vxlan interfaces, which isn't enabled in the kernel. Is enabling it possible?

Thanks!

**chewitt** · March 25, 2019 at 9:48 PM Official Post

What's the use-case for needing LE devices in a swarm?

Snapcase · March 30, 2019 at 9:14 PM

Hi,

Swarms are native clustering for Docker and useful if you have multiple hosts running on seperate machines. It allows running containers on different machines to share resources like networks, secrets, etc and well as basic load balancing etc.

They're useful in any case where you have more than one docker host. In my particular instance, it was because LE and my other docker host are relatively low powered machines and I wanted to move some containers around for performance reasons.

It's built into docker so the current official add-on supports creating and joining swarms and gives the appearance that everything is working when in reality the underlying docker network doesn't work.

I have no idea what the implications or risks are of adding kernel modules, and I've worked around my particular problem. It also looks like I'm the only one who has ever brought this up...

If it's not doable, because it's a standard docker feature, perhaps the fact swarms aren't supported could be added to the add-on text or wherever is appropriate.

At the very least this thread will save people from trying to debug this for days like I did

Thanks for your help

Kris

DarkHorse · February 26, 2024 at 3:47 PM

Ah bummer... just got LibreElec running on my old Beelink GT1 device that was gathering dust in a drawer... was hoping to add it as an additional node to my docker swarm, but it looks like I've run into the issue described here in this thread where there is no vxlan module available. As such, it is unable to participate as a node in the docker swarm. Oh well, I'll purpose it to some other task.

cheers.

**chewitt** · February 27, 2024 at 6:13 AM

Code

projects/Allwinner/linux/linux.aarch64.conf:CONFIG_VXLAN=m
projects/Allwinner/linux/linux.arm.conf:CONFIG_VXLAN=m
projects/Amlogic/linux/linux.aarch64.conf:# CONFIG_VXLAN is not set
projects/Generic/linux/linux.x86_64.conf:CONFIG_VXLAN=m
projects/NXP/devices/iMX6/linux/linux.arm.conf:CONFIG_VXLAN=m
projects/NXP/devices/iMX8/linux/linux.aarch64.conf:CONFIG_VXLAN=m
projects/Qualcomm/devices/Dragonboard/linux/linux.aarch64.conf:CONFIG_VXLAN=m
projects/RPi/devices/RPi/linux/linux.arm.conf:# CONFIG_VXLAN is not set
projects/RPi/devices/RPi2/linux/linux.arm.conf:CONFIG_VXLAN=m
projects/RPi/devices/RPi4/linux/linux.aarch64.conf:CONFIG_VXLAN=m
projects/RPi/devices/RPi5/linux/linux.aarch64.conf:CONFIG_VXLAN=m
projects/Rockchip/devices/RK3288/linux/default/linux.arm.conf:CONFIG_VXLAN=m
projects/Rockchip/devices/RK3328/linux/default/linux.aarch64.conf:CONFIG_VXLAN=m
projects/Rockchip/devices/RK3399/linux/default/linux.aarch64.conf:CONFIG_VXLAN=m
projects/Samsung/linux/linux.arm.conf:CONFIG_VXLAN=m

Display More

It looks like Amlogic (and RPi 0/1) are the only kernels we ship that don't have VXLAN support enabled. That can be changed.

DarkHorse · February 27, 2024 at 3:42 PM

Ah, great! OK, I enabled the VXLAN option in the Amlogic config, did a build, modified /etc/modules-load.d to include vxlan.conf to autostart the module, and now it is working great as a node in the swarm! Thanks!

Here is a picture of the docker swarm as shown from Portainer... "quadra5" at the bottom is the Beelink GT1 with 8 cpu cores.

Sweet!

**chewitt** · February 28, 2024 at 9:21 AM

Nice. I'll add the module change to the next kernel defconfig bump that I push.

scotts · May 22, 2024 at 12:52 AM

Sorry to resurrect this topic, but I've kinda been following the same footsteps with having a docker worker node running in the background behind libreelec along with my other raspberry 4s. After saving vxlan.conf in /etc/modules-load.d and adding the node to the swarm if I replicate a container to the LibreELEC node I get the following error:

"Extension bpf revision 0 not supported, missing kernel module?"

This is the result from a different node called toolbox running Raspberry Pi OS lite. If I try to replicate a container it replicates fine on other boxes just not LibreELEC.

pxaai0vdgxyyzcxw3hb3j6c0d proxy_homepage.2 ghcr.io/gethomepage/homepage:latest@sha256:43a3ee88abe3b37c64bc52ea93da01c3dcb4a332a953bcd7f438c8d7328d3947 toolbox Running Running 49 minutes ago
ob0b49rercm1l7zlui0q8jgpi proxy_homepage.3 ghcr.io/gethomepage/homepage:latest@sha256:43a3ee88abe3b37c64bc52ea93da01c3dcb4a332a953bcd7f438c8d7328d3947 archtop Running Running 49 minutes ago
wz5bv87g06fhrpg3w43zaxgpd \_ proxy_homepage.3 ghcr.io/gethomepage/homepage:latest@sha256:43a3ee88abe3b37c64bc52ea93da01c3dcb4a332a953bcd7f438c8d7328d3947 LibreELEC Shutdown Rejected 2 hours ago "network sandbox join failed: subnet sandbox join failed for "10.13.0.0/16": could not install mangle rule: (iptables failed: iptables --wait -t mangle -A OUTPUT -p udp --dport 4789 -m bpf --bytecode 7,32 0 0 4294963252,7 0 0 0,64 0 0 4,116 0 0 8,21 1 0 4105,6 0 0 0,6 0 0 4294967295 -j MARK --set-mark 13681891: Warning: Extension bpf revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.
(exit status 1))"

I understand Docker Swarms are outside the scope of the LibreELEC project and maybe I would have better luck asking on dedicated docker forum, but I just happened to see that the OP of the thread is doing a similar project to myself.