Potential conflict between connman-vpn and openvpn services

dpapavas · February 29, 2024 at 6:10 PM

I've set up an openvpn-based connection, based on .config/system.d/openvpn.service.sample. It works, but I periodically get log messages such as these

Quote

Feb 29 19:12:22 pi4b openvpn[1064]: [UNDEF] Inactivity timeout (--ping-restart), restarting
Feb 29 19:12:22 pi4b openvpn[1064]: SIGUSR1[soft,ping-restart] received, process restarting
Feb 29 19:17:22 pi4b openvpn[1064]: TCP/UDP: Preserving recently used remote address: [AF_INET]195.178.172.188:1194
Feb 29 19:17:22 pi4b openvpn[1064]: NOTE: setsockopt TCP_NODELAY=1 failed
Feb 29 19:17:22 pi4b openvpn[1064]: UDP link local: (not bound)
Feb 29 19:17:22 pi4b openvpn[1064]: UDP link remote: [AF_INET]195.178.172.188:1194
Feb 29 19:18:22 pi4b openvpn[1064]: [UNDEF] Inactivity timeout (--ping-restart), restarting
Feb 29 19:18:22 pi4b openvpn[1064]: SIGUSR1[soft,ping-restart] received, process restarting

Display More

and the connection is unstable, i.e. some time after starting the openvpn service the device seems to lose internet connectivity. I have a hunch this may be due to the fact that the connman-vpn service is running at the same time, even though I have no active configuration for it. It seems though that onve I stop it, the above messages stop and the connection is more stable.

The problem is that I can't seem to disable this service. It is shown as disabled, but is started nevertheless on next boot. I think that's because all these system services reside in /usr/lib/systemd/system, which is on a readonly partition, so that systemctl disable can't remove the symlinks that bind it to the multi-user target.

Is there some way to disable the connman-vpn service?

**chewitt** · March 1, 2024 at 1:58 AM

stop > disable > mask ?

ConnMan is deliberately compiled with no support for OpenVPN connections so ConnMan-VPN does not attempt management of NIC interfaces that are created by OpenVPN.

dpapavas · March 1, 2024 at 11:20 AM

First time I've heard of systemctl mask, I must confess. Thanks, I'll give it a try.

I'm afraid though, that this is not the root of my problem, as I see now that I've been getting these "ping restarts" even without connman running. I've also been getting the following weird issues, seemingly only when openvpn is running, although it's hard to tell. Out of the blue I get a disconnect with

Quote

Mar 01 12:20:16 pi4b iwd[420]: Received Deauthentication event, reason: 0, from_ap: false
Mar 01 12:20:16 pi4b connmand[532]: wlan0 {RX} 4581242 packets 971720702 bytes
Mar 01 12:20:16 pi4b connmand[532]: wlan0 {TX} 2020293 packets 384926147 bytes
Mar 01 12:20:16 pi4b connman-vpnd[412]: wlan0 {update} flags 4099 <UP>
Mar 01 12:20:16 pi4b connmand[532]: wlan0 {update} flags 4099 <UP>
Mar 01 12:20:16 pi4b connman-vpnd[412]: wlan0 {newlink} index 3 address 6C:21:A2:38:08:98 mtu 1500
Mar 01 12:20:16 pi4b connman-vpnd[412]: wlan0 {newlink} index 3 operstate 2 <DOWN>
Mar 01 12:20:16 pi4b connmand[532]: wlan0 {newlink} index 3 address 6C:21:A2:38:08:98 mtu 1500
Mar 01 12:20:16 pi4b connmand[532]: wlan0 {newlink} index 3 operstate 2 <DOWN>

Display More

and then it reconnects with a weird IP, which it obviously doesn't get from my router's DHCP.

Quote

Mar 01 12:20:52 pi4b connmand[532]: wlan0 {add} address 169.254.35.186/16 label wlan0 family 2
Mar 01 12:20:52 pi4b connmand[532]: wlan0 {add} route 169.254.0.0 gw 0.0.0.0 scope 253 <LINK>
Mar 01 12:20:52 pi4b connmand[532]: wlan0 {add} route 0.0.0.0 gw 0.0.0.0 scope 253 <LINK>
Mar 01 12:20:52 pi4b connmand[532]: wlan0 {add} route 0.0.0.0 gw 0.0.0.0 scope 253 <LINK>

I had reached out to Radxa, about the firmware, and one of their devs said I should make sure to disable WPA3, as it may give their devices connectivity problems. If you know how I can do that in LE, let me know.

**chewitt** · March 1, 2024 at 12:22 PM

169.254.35.186/16 is a link-local (self-assigned) address which normally indicates something has failed at layer2 and prevents layer3 functions like DHCP from succeding .. i.e. you fell off the network.

Issues are unlikely to be from ConnMan as all it really does is instruct the wireless daemon (iwd) to start/stop connections. It might be worth seeing if iwd itself is the issue by doing a clean compile of LE with ye olde wpa_supplicant not iwd, the choice is configured in the distributions/LibreELEC/options file IIRC.

dpapavas · March 2, 2024 at 1:37 PM

Thanks for the pointer! Before trying that, I thought I'd try to set up my VPN connection with WireGuard. I've created a configuration file, but can't seem to be able to have it show up under connmanctl services. Here's the config file:

Quote

pi4b:~ # cat .config/wireguard/protonvpn.conf
[provider_wireguard]
Type = WireGuard
Name = protonvpn
Host = 195.178.172.188
WireGuard.Address = 10.2.0.2/24
WireGuard.ListenPort = 51820
WireGuard.PrivateKey = eON...vGw=
WireGuard.PublicKey = GB...FDms=
WireGuard.DNS = 10.2.0.1
WireGuard.AllowedIPs = 0.0.0.0/0
WireGuard.EndpointPort = 51820
WireGuard.PersistentKeepalive = 25

Display More

The connman-vpn service is running normally. I can't see any sort of error in the logs. Any idea what might be wrong, or how I could go about troubleshooting it?

dpapavas · March 2, 2024 at 2:09 PM

My configuration file was misnamed. It had a .conf extension, instead of the correct .config. It's working now. Let's see if the situation is any better than it was with openvpn.

**chewitt** · March 2, 2024 at 2:36 PM

I can't speak for that service, but WireGuard generally delivers much better performance than OpenVPN does.

dpapavas · March 2, 2024 at 5:36 PM

It does seem to work better. One issue I have though, is that if the connection is lost once I've started the service, connman reconnects to my wireless connection. And since the wireguard service has exited succesfully, even if I make it RequiredBy/Before another service, say kodi there doesn't seem to be a way to make sure that VPN always stays connected (unless perhaps explicitly and manually disconnected).

**chewitt** · March 3, 2024 at 9:15 AM

The wireguard.service example has:

Code

[Service]
Type=oneshot

So once it fails, that's it.

The openvpn.service example has:

Code

[Service]
Type=forking
Restart=always
RestartSec=15

Do some reading on systemd and experiement..

dpapavas · March 3, 2024 at 12:24 PM

I don't think it's just a matter of systemd service configuration. The wireguard service just issues a connmanctl connect vpn_name and exits, yes, so it has no way of monitoring the connection. In fact, due to connman pretty much always returning a succesful exit status, it has no way of knowing whether the connection was established at all. It's always successful.

The openvpn service forks the process that actually establishes the connection and maintains it, so if something happens to the connection, this process fails and the whole service fails. One can then set this up, so that it also stops dependent services.

The only ways I see of getting something similar with Wireguard, as handled by connman, is to either somehow use connman monitor services inside the wireguard service, to monitor the connection status, or to use the virtual network interface device that is created by udev and systemd.

I've done the latter. Briefly, assume you get a wg0 network interface when the connection is established, then you also get a sys-devices-virtual-net-wg0.device systemd unit when that happens, which goes away if the connection is dropped for whatever reason. If you then want, say, kodi.service to only run when the VPN connection is up and running, you can add to its [Unit] section:

Code

Requires=wireguard.service
After=wireguard.service
BindsTo=sys-devices-virtual-net-wg0.device
After=sys-devices-virtual-net-wg0.device

This starts the wireguard service to (hopefully) establish the connection, but also waits for the connection to appear (via BindsTo on the virtual device) before starting Kodi. If then the connection is dropped, Kodi is also stopped.

This is reliable, but still has a kink: Due to the way wireguard.service works, it remains "active", since it has no way of knowing that the connection has been dropped. So if you want to restart Kodi, you need to restart the wireguard.service manually first, because otherwise the dependent Kodi service thinks it's already started and never restarts the connection. If you only want the wireguard.service to start in tandem with Kodi, you can add the following to its [Unit] section:

Code

StopWhenUnneeded=true

Then if the connection drops, Kodi is stopped, which makes the wireguard.service "unneeded", so it is stopped too.

dpapavas · March 6, 2024 at 4:47 PM

I tried a recompile with wpa_supplicant, as suggested and have been using it for the past few days. The connection has certainly been more stable. I still get disconnects from my router

Quote

Mar 03 21:59:08 pi4b wpa_supplicant[513]: wlan0: CTRL-EVENT-DISCONNECTED bssid=c0:9f:51:23:10:8e reason=3

but now it reconnects without issues. With iwd, I'd get failures such as the one described with the link-local address and also more curious cases, where Kodi would ask for a passphrase again (perhaps the MAC address changed on reconnect?). I asked Radxa, if they know of any issues with iwd, but they're using NetworkManager, for which iwd support is still experimental, so they use wpa_supplicant by default.

Not sure if wpa_supplicant would be the appropriate choice for all Pi 4B users, but I though I'd mention my experience.

**chewitt** · March 7, 2024 at 10:18 AM

wpa_supplicant is the Xorg of the WiFi world; it's used everywhere, it works, but it's architecturally hideous

iwd is the Wayland that will replace it everywhere, soon, but it's still young(er) and maturing (rapidly).

The latest iwd version bump that went into nightlies a couple of days ago has a pile of fixes for connection issues; the devil is always in the detail but might be worth seeing if anything improves. If it doesn't the investigation process requires you to run iwd in debug mode to capture verbose output traces for bad behaviours/failures and then report them upstream.

NB: LE has long been an advocate for newer and better code that seeks to displace older stuff so we'll be sticking with iwd. It's good to compare with wpa though, if only to help point fingers in the right direction.

dpapavas · March 7, 2024 at 12:44 PM

I think I'll give iwd another try when I switch over to LE 12 and see if anything has improved. If not, I'll see about debugging it.