WiFi 'Invalid-Key' on LibreELEC 11/12 (inc. workaround)

Ok, Ok, I'm playing with words here. I said fixed, not a solution.

More specifically, I have identified the sequence of commands to run so that you will never see Invalid-Key again.

Still there are a few things going on here and first and foremost, I believe the problem with Invalid-Key I am having relates to my 10-year-old NightHawk R7000 router running FreshTomato Firmware 2024.1 K26ARM USB AIO-64K with Linux kernel 2.6.36.4brcmarm and Broadcom Wireless Driver 6.37.14.126 (r561982). My settings ('Advanced'-->'Wireles') can be seen here.

Additionally, the Invalid-Key issue was particularly heinous because I have a half-dozen storage-*.mounts in /storage/.config/system.d/. They attempt to start/mount prior to autostart.sh. Of course they are bound to network-online.service. So, I made sure that they were defined with BindsTo=network-online.target. Still on LE 11.0x I would sporadically get the Invalid-Key. I would more often need to restart LE and as a result not have my storage mounts mounted and then need to reboot again and again, until I could connect. I have addons, backgrounds, favorites, iconds, backups, and more mounted this way so when this happened, Kodi looked pretty gnarly. Barely useable and required a reboot and flip a coin if it connected this time.

The problem persisted on LE 12.01 although admittedly significantly less. I played around with connmanctl, and in time noticed the errors that made me look at my router. Again, its FreshTomato and honestly I can't find anyone who knows what all that crap on 'Advance' --> 'Wireless' page of the FreshTomato UI means. So, I went to ChatGPT. The settings above are what it suggested for compatibility with a RPi5.

Still, No love! Invalid-Key persisted.

After more digging I realized my autostart.sh script really needed updating. I was explicitly enabling each of my storage mounts with 'systemctl enable' one at a time and figured it was easier to do something like, 'for m in $(find /storage/.config/system.d/ -iname storage-*.mount -exec basename "{}" \;); do systemctl enable $m; done'. It was then that I realized I could start them the same way. I had previously just been forcing a reboot.

All this is what started me down the rabbit hole of connmanctl and iwctl. I'll spare you all the gory details, but below is the result of tinkering every night since Friday. In the process I discovered the Nightly build from 10.05.2024. Armed with it and the script below, I am connecting every time I reboot. I have NOT been able to get it to fail.

Let me be clear, I get errors from my storage mounts about 50% of the time and I may still not be able to "automatically" connect during a reboot from time to time due to external factors like distance and noise or interference from other electronic devices with various other Wi-Fi tech. Or needing to reboot my router or something. But after every reboot I am connected each time.

Also, manual disconnects and reconnects from within the LibreELE AddOn seems to be working. But I am not pushing my luck there because during all of the trial and error, there were times when the LE Addon wouldn't open for some odd reason.

While I still believe the issue lies with my router's age and custom rom, I have been second-guessing myself about that because of the drastic swing in behavior of the script from LE 12.01 to LE Nightly's 10.05.24 & LE 10.8.24. My router isn't changing. Why is the behavior changing? Something relating to configuration or versioning in LE? Maybe.

Anyway, this works for me, I hope it works for you. Here you are...

#!/bin/sh
# Wi-Fi Auto Connect Script
# This script automatically attempts
# to connect to the specified SSID. If
# the connection fails after the first
# two attempts, it restarts connman
# and retries two additional times.
# The Wi-Fi credentials (SSID and password)
# can optionally be configured in the file:
#   /storage/.config/connman_main.conf
#
# Sample content for the configuration file:
#   [service_wifi]
#   Type = wifi
#   Name = your_SSID
#   Passphrase = your_password
#
# Modify 'Name' and 'Passphrase' to 
# match your SSID and Wi-Fi password.
# In practice, I've only gotten the 
# SSID favorited this way
# Global Variables

SSID="Your_SSID"
SECRET="Your_Password"
SLEEP_TIME=3  # Time to pause between retries (in seconds)
LOGFILE="/storage/.config/autostart.log"
CONNECTED="FALSE"

# Truncate the log file at the beginning of the script
> "$LOGFILE"

# Logging function to prepend timestamp to each log entry
log() {
    while IFS= read -r line; do
        echo "$(date '+%a %b %-d %H:%M:%S %Z %Y') - $line" >> "$LOGFILE"
    done
}
# Redirect stdout and stderr to the log function
exec > >(log) 2>&1

# Start Processing of AutoStart script.
echo "AutoStart script is starting."

# Perform an initial scan
echo "Scanning for Wifi Networks."
iwctl station wlan0 scan
connmanctl scan wifi

# Function to get the service identifier dynamically based on the SSID
get_service_id() {
    connmanctl services | grep "$SSID" | awk '{print $3}'
}

# Function to attempt connection to the Wi-Fi network
attempt_connection() {
    SERVICE_ID=$(get_service_id)
    
    if [ -z "$SERVICE_ID" ]; then
        echo "Error: Unable to find service identifier for '$SSID'. Rescanning..."
        iwctl station wlan0 scan
        connmanctl scan wifi
        return 1
    fi
    
    # Establish connection to Wifi on wlan0
    ERROR=$(/usr/bin/iwctl station wlan0 connect $SSID --passphrase=$SECRET 2>&1)
    timeout=$(($(date +%s) + 15))
    until [ -n "$(ip -4 route get 8.8.8.8 2>/dev/null | awk {'print $7'} | tr -d '\n')" ] || [ $(date +%s) -gt $timeout ]; do sleep $SLEEP_TIME; done
    journalctl | grep "iwd\["
    
    ip=$(iwctl station wlan0 show | grep "IPv4 address" | awk '{print $3}')
    if [ ${#ip} -gt 0 ]; then
        echo "Connected to '$SSID'. IP Address: $ip."
        CONNECTED="TRUE"
        return 0
    else
        echo "Failed to connect to '$SSID'."
        return 1
    fi
}

# Function to handle the retry logic with a specified max retries
retry_connection() {
    local MAX_RETRIES=$1
    local retry_count=0

    while [ $retry_count -lt $MAX_RETRIES ]; do
        echo "Attempting manual connection to Wi-Fi (Attempt $((retry_count + 1)) of $MAX_RETRIES)..."

        # Try to find the SSID and connect
        attempt_connection
        if [ $? -eq 0 ]; then
            return 0
        fi
        retry_count=$((retry_count + 1))
        sleep $SLEEP_TIME
    done
    return 1
}

# Wait for network to startup on its own, checking every 3s for a max of 15s.
echo "Waiting for network to connect (First Attempt)."
timeout=$(($(date +%s) + 15))
until [ -n "$(ip -4 route get 8.8.8.8 2>/dev/null | awk {'print $7'} | tr -d '\n')" ] || [ $(date +%s) -gt $timeout ]; do sleep $SLEEP_TIME; done

ip=$(iwctl station wlan0 show | grep "IPv4 address" | awk '{print $3}')
if [ ${#ip} -eq 0 ]; then

    # Perform another scan
    echo "Not connected. Performing another Wifi scan."
    iwctl station wlan0 scan
    connmanctl scan wifi
    
    # Try to connect and rescan up to 3 times before clearing the cache
    retry_connection 1
    if [ $? -ne 0 ]; then
        echo "Failed to connect after 2 attempts."
        # Restart the connmanctl service to ensure fresh cache is generated
        echo "Restarting connmanctl service..."
        connmanctl disable wifi
        connmanctl enable wifi
        iwctl station wlan0 scan
        connmanctl scan wifi
        retry_connection 2
    fi
    [ "$CONNECTED" = "FALSE" ] && {
        echo "Network connection could not be established."
        exit 1
    }
else
    echo "Connected to '$SSID'. IP Address: $ip."
fi

# Make sure network shares are mounted if not present when this is run.
[ ! -d /storage/.config/system.d/multi-user.target.wants ] && {
	mountcount=$(find /storage/.config/system.d -type f -iname "storage*mount" | wc -l)
	[ $mountcount -gt 0 ] && {
		# Configuring CIFS Mount points to auto-start at device boot.
		echo "Configuring CIFS Mount points to auto-start at device boot...."
		for m in $(find /storage/.config/system.d/storage*mount -exec basename "{}" \;); do systemctl enable $m; done

		# Mounting CIFS Storage Mount Points
		echo "Mounting CIFS Storage Mount Points..."
		for m in $(find /storage/.config/system.d/storage*mount -exec basename "{}" \;); do systemctl start $m; done

		echo "Processing complete. CIFS Mount points have been enabled and started."
	} || {
		echo "No CIFS mount point files to process."
	}
} || {
	# Mounting CIFS Storage Mount Points
	echo "Mounting CIFS Storage Mount Points..."
	for m in $(find /storage/.config/system.d/storage*mount -exec basename "{}" \;); do systemctl start $m; done

	echo "Processing complete. CIFS Mount points have been started."
}
echo "AutoStart script complete!"

NOTE: /storage/.config/connman_main.conf .. It doesn't work. I reference it and found that it worked to save my WiFi connection as a favorite and it would show up in a scan from the LibreELEC Addon and even I saw evidence in the script logs that iwctl couldn't find the SSID but connected anyway. So I suspect it was because of this file. Don't know but suspect it plays some role.

[reserved]

Point taken, but I admitted playing with words here. I said fixed, not a solution. I don't have to repeatedly reboot to reconnect and map drives. I also don't see Invalid-Key when it doesn't connect. I will see Operation Failed or Input/output errors but not Invalid-Key. I've been using LE since version 9 (You've aged gracefully btw). I've always seen Invalid-Key. I just usually tolerate and work around it. Again, I admit, that is indeed what I am doing here. Working around it. But something in what I did has suppressed that specific error.

However, since this all started on Friday, my router is 87 i.n.c.h.e.s away from the RPI5. Signal strength is 90+. Saturday night, while using 12.01, I successfully connected and set up LE during First Run. I even successfully connecting, a few times after that. At some point after that and a significant amount of tinkering, I could not get it to connect for anything. Not after a reboot, not after a reconnect, not after deleting the connection in LE addon and recreating it. Not from the terminal window. I didn't fix that. I only got it working again, by reformatting the SDCard and re-imaging LE. That's when found ``Nightly 12 - 10-05-2024` . With this build, on Sunday night, I rebooted 25x and it connected on first attempt (including mounting storages). All testing was done with only First Run 'vanilla' being configured. No addons added. No configuration changes. Not even changing the Country / Region / Rev settings in LE. I'm open to being the router and signal strength, I just don't see how, having experienced that. Because the only thing that changed before and after those (success and failure) sessions was my script and the version of LE I was running. So, I need someone to explain that to me.

I'm not a developer. I can script. I am a Lead Engineer. I will try whatever you want.

Wow, you're censoring 'inch(es)'? Wonder what a-hol3 made a necessity.

Ok, I reduced the autostart.sh script to bare minimum. This is the debug script. I left the CIFS storage mount files in /storage/.config/system.d/. Since the storage mounts are attempted prior to the execution of autostart.sh, this enabled me to more easily see if LibreELEC was establishing a connection or the autostart.sh script was establishing the connection. If LibreELEC failed to make a connection the storage mounts would fail with pretty obnoxious error message during boot up. No error during boot meant LibreELEC made the connection.

That was my naming convention. Success or Failure zip files mean LibreELEC succeeded or failed.

However, in combination with the script below, I could not get LibreELEC to fail to establish a connection using the Nightly Build from 10.08.2024. Every. Single. Time. I tried multiple times. At least 20x reboots. Not one single failed connection. Either the script made the connection or LibreELEC did. This basically means the script never fails to establish a connection because a) there is no check for an existing connection so it can exit. b) there is no retry logic in the script (aside from the retries built into iwd and connmand).

I've attached the logs for your review. There is an 'invalid-key' in it though; so, it is indeed a workaround not a fix.

Now, I just need to append a quick test for a valid IP Address and the line of code below and I am golden. My mounts will be established as well.

for m in $(find /storage/.config/system.d/ -iname storage-*.mount -exec basename "{}" \;); do systemctl start $m; done

Here's the abridged, debug version of the script:

#!/bin/sh

# Global Variables
SSID="serendrewpity"
SECRET="BoogerA!ds"
SLEEP_TIME=3  # Time to pause between retries (in seconds)
LOGFILE="/storage/.config/autostart.log"

# Truncate the log file at the beginning of the script
> "$LOGFILE"

# Logging function to prepend timestamp to each log entry
log() {
    while IFS= read -r line; do
        echo "$(date '+%a %b %-d %H:%M:%S %Z %Y') - $line" >> "$LOGFILE"
    done
}

# Redirect stdout and stderr to the log function
exec > >(log) 2>&1

# Start Processing of AutoStart script.
echo "AutoStart script is starting."

# Start Processing of AutoStart script.
echo "Stopping the default ConnMan and Internet Wireless Daemon services."
systemctl stop connman 
systemctl stop iwd
kill -9 $(pgrep connmand) $(pgrep iwd)

# Starting IWD.
echo "Starting the Internet Wireless Daemon service."
nohup /usr/lib/iwd --debug > /storage/.config/iwd_debug.log 2>&1 & 
sleep 2

# Starting ConnMan.
echo "Starting the ConnMan daemon service."
nohup /usr/sbin/connmand --debug > /storage/.config/connman_debug.log 2>&1 &
sleep 2

# Perform an initial scan
echo "Scanning for Wifi Networks."
iwctl station wlan0 scan
connmanctl scan wifi

# Attempt to connect to Wifi on wlan0
echo "Attempting to connect to Wifi SSID on wlan0."
ERROR=$(/usr/bin/iwctl station wlan0 connect $SSID --passphrase=$SECRET 2>&1)
timeout=$(($(date +%s) + 25))
until [ -n "$(ip -4 route get 8.8.8.8 2>/dev/null | awk {'print $7'} | tr -d '\n')" ] || [ $(date +%s) -gt $timeout ]; do sleep $SLEEP_TIME; done
journalctl | grep -E 'iwd\[|connmand\['

# AutoStart script finished.
echo "AutoStart script finished."

The stanza for 'journalctl' in the script is pretty slick. It captures every single command from 'iwd' or 'connmand' from the start of boot.