NFS issue with share from Debian kernel 6.x

If reverting the kernel change on your NAS eliminates the issue, then you need to bisect the changes between the working and non-working kernel versions to pinpoint the problem commit; then report this to the upstream kernel maintainers to pursue a fix. Before sending emails to mailing lists I'd suggest you also check that fixes are not already merged into newer (and current development) kernels to avoid reporting a problem that's already resolved.

Hi there. First, thank you very much chewitt for your countless contributions. Much appreciated!

I encounter a similar issue:

I ran libreelec 9 on Wetek Play2 and accessed movies on diffent NFS-Shares. This worked fine until I upgraded Wetek Play2 to libreelec 11.0.1. After this upgrade I ran into the same issues as descibed by Fietspomp. Browsing NFS share work, running the movies as well until i begin to skip, fastforeward or until the movie played for some time. Running the same movies from USB-stick works well, so this does not seem to be a decoding issue.

Debugging the NFS layer, I tried both, configuring NFS-Shares in the Kodi GUI as well as on libreelec OS level (https://wiki.libreelec.tv/how-to/mount_network_share). Both resulted in the same problems. When mounting the share with systemd the options used don't look suspicious to me: (i tried also both, nfsv3 and nfsv4)

# mount
...
192.168.1.99:/data/downloads on /storage/downloads type nfs4 (rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.1.100,local_lock=none,addr=192.168.1.99)

On the NFS-server side, nothing changed. I google for similar issues and found this RedHat knowledge article: https://access.redhat.com/solutions/5953561

For those without access to it, it basically says:

• After updating to kernel-4.18.0-240.el8 NFS read performance is far lower than previous RHEL8 releases.
• Downgrading to kernels prior to kernel-4.18.0-240.el8 causes performance to resume to previous levels

• A commit was included into Red Hat Enterprise Linux 8.3 that lowered the default value for read_ahead_kb. This reduces the amount of pre-fetching done during read procedures which can impact large streaming reads.
• Historically, read_ahead_kb has been equal to the negotiated rsize for the share at time of mount multiplied by fifteen.
• affected is kernel-4.18.0-240.el8 and later

The default read_ahead_kb seems to get set at only 128 KiBs on libreelec. I was able to verify this on my device:

# cat /sys/class/bdi/$(mountpoint -d /storage/downloads/)/read_ahead_kb
128

I really don't understand enough of this value to decide if this is really a problem or not. I tried to set the read_ahead_kb value to the negotiated (betwenn nfs client and server) value:

# echo 1048576 > /sys/class/bdi/$(mountpoint -d /storage/downloads/)/read_ahead_kb
# cat /sys/class/bdi/$(mountpoint -d /storage/downloads/)/read_ahead_kb
1048576
# echo 3 > /proc/sys/vm/drop_caches

Unfortunately, this didn't help. I wonder how I could set another value in a persistent way anyways...

Does anyone has an idea if this is really a problem or am I looking to solve my problem in the wrong place?

I don't think the RHEL article is relevant because the 4.18 kernels are ancient (even though RH backports a ton to them). That also doesn't correlate with the OP who broke his setup by updating the kernel on his OMV install.

You can also look at Kodi changes for NFS and filesystem caching. I have fuzzy recall that there are some between K19 and K20, but as I never use NFS myself (as SMB works fine) I never pay any attention to them. Some differences to an LE9 install would be expected since the entire codebase is different, but we aren't seeing general user whinging about NFS issues with LE11 and that usually means the reported issues are more localised to a specific other software (are you also using the same OMV version?) or something else niche/environmental.

Thank you for sharing your thoughts chewitt. I was able to verify that my issue is not related to the read_ahead_kb value of nfs.

I was able to set this value on libreelec with an udev-rule:

# cat /etc/udev/rules.d/99-nfs.rules

SUBSYSTEM=="bdi", ACTION=="add", PROGRAM="/bin/awk -v bdi=$kernel 'BEGIN{ret=1} {if ($4 == bdi) {ret=0}} END{exit ret}' /proc/fs/nfsfs/volumes", ATTR{read_ahead_kb}="16000"

After reboot, the value is set correctly, but unfortunately the issues with playback persist.

# reboot

...

# cat /sys/class/bdi/$(mountpoint -d /storage/downloads/)/read_ahead_kb

16000

I will go on with my debugging and not spam this thread any further

EDIT:
I found, that the issues are network related. Video playback with 1Gbit connection worked fine, Wifi worked not bad when i put my Wetek Play2 box very close to the Wifi router. Then I just followed this guide https://www.kodi-guide.com/how-to-fix-kodi-buffering/ to use more buffering and my box again works like a charm! (even better )

Why the wifi performance dropped after upgrading to Libreelec 11.0.1 is beyond my understanding and perhaps it's just a coincidence. I'm very happy that everything works again. Thanks to everyone involved in bringing Libreelec 11 to Wetek Play2 eventhough its an old box. It's much appreciated!