RPi 4 2 GB losing USB - Firmware issue?

    Quote from camelreef

    IIUC, the RPi OS that worked well was using the Tue 25 Jan 14:30:41 UTC 2022 (1643121041) firmware.

    This is NOT a firmware, it's a bootloader, stored in RPi 4B's EEPROM. Until it's upgraded/downgraded automatically by OS or manually, it stays there even you boot from another microSD card or HDD.

    The firmware files are stored on boot media's partition (.elf files etc.). So they are different with every boot media used.

    I don't know which firmware version uses latest LE Nightly but you can compare them with files on RPi OS which works for you.

    Quote from ghtester

    This is NOT a firmware, it's a bootloader, stored in RPi 4B's EEPROM. Until it's upgraded/downgraded automatically by OS or manually, it stays there even you boot from another microSD card or HDD.

    The firmware files are stored on boot media's partition (.elf files etc.). So they are different with every boot media used.

    I don't know which firmware version uses latest LE Nightly but you can compare them with files on RPi OS which works for you.

    I learn every day!

    OK, I'll look at the boot message, generate md5sums on both systems and report back.

    Could I steal the elf files from RPi OS and use them on my LE11 nightly, or is this the stupidest thing you've heard today?

    Quote from camelreef

    Could I steal the elf files from RPi OS and use them on my LE11 nightly, or is this the stupidest thing you've heard today?

    I would say it's not an absolute nonsense - it might work but of course it's not guaranteed. All versions (bootloader, firmware and OS) should match together (to be at supported levels). You can test it or wait, I suppose that LE developers are going to upgrade the RPi 4B's firmware as well but it may take some time.

    Quote from ghtester

    I would say it's not an absolute nonsense - it might work but of course it's not guaranteed. All versions (bootloader, firmware and OS) should match together (to be at supported levels). You can test it or wait, I suppose that LE developers are going to upgrade the RPi 4B's firmware as well but it may take some time.

    start.elf and fixup.dat, right?

    Nico

    Kernel+firmware bump PR just got merged, the next nightly will include that.

    LE uses the start_x variant of the firmware, http://start4x.elf/fixup4x.dat renamed to http://start.elf/fixup.dat, so it's best to test with "start_x=1" in config.txt on RPiOS to make sure the exactly same firmware is used.

    Or you can download http://start4x.elf/fixup4x.dat from here https://github.com/raspberrypi/firmware , rename them to http://start.elf/fixup.dat and put them on the LE /flash partition.

    so long,

    Hias

    [deleted post content]

    scratch that, 1D10T error... See next post!

    OK, new fixup/start files from the next branch (https://github.com/raspberrypi/firmware/tree/next/boot)!

    It boots!

    With new fixup/start:

    Code
    kodiplayer:~ # dmesg | grep firmware
[    0.110299] raspberrypi-firmware soc:firmware: Attached to firmware from 2022-01-25T13:48:40, variant start_x
[    0.113642] raspberrypi-firmware soc:firmware: Firmware hash is 94562b1518ca82ece28042cca1e5cdbbb43c8bda

    So newer firmware than the original (2022-01-06T15:40:20), which is good.

    But the remote CIFS client still drops. Oh well...

    I'll wait for the full package, including kernel, inside the next nightly!

    11.0-nightly-20220204, right?

    Thanks again!

    Nico

    Edited 2 times, last by camelreef: Merged a post created by camelreef into this post. (February 3, 2022 at 5:52 PM).

    Quote from camelreef

    But the remote CIFS client still drops. Oh well...

    CIFS issue probably has nothing to do with firmware. I assume you are using Samba on "kodiplayer" and it's crashing from what is noted above.

    This to me suggests the client "grabber"(?) is the one with the issues:

    Code
    smb2_sendfile_send_data: sendfile failed for file Transit/Incoming/The.Book.of.Boba.Fett.S01E06.2160p.WEB-DL.x265.10bit.HDR.DDP5.1.Atmos-BobarBiriyani[rartv]/The.Book.of.Boba.Fett.S01E06.Chapter.6.2160p.WEB-DL.DDP5.1.Atmos.HDR.HEVC-BobarBiriyani.mkv (Connection reset by peer) for client ipv4:10.25.25.1:44272

    Connection reset by peer is initiated by the peer -- they send an RST packet. Seems like a network issue perhaps, because 7 minutes earlier CIFS is "reconnecting" -- what was the cause of that? Maybe on the client see if vers=3.0 works on the mount options?

    Quote from frakkin64

    CIFS issue probably has nothing to do with firmware. I assume you are using Samba on "kodiplayer" and it's crashing from what is noted above.

    This to me suggests the client "grabber"(?) is the one with the issues:

    Code
    smb2_sendfile_send_data: sendfile failed for file Transit/Incoming/The.Book.of.Boba.Fett.S01E06.2160p.WEB-DL.x265.10bit.HDR.DDP5.1.Atmos-BobarBiriyani[rartv]/The.Book.of.Boba.Fett.S01E06.Chapter.6.2160p.WEB-DL.DDP5.1.Atmos.HDR.HEVC-BobarBiriyani.mkv (Connection reset by peer) for client ipv4:10.25.25.1:44272

    Connection reset by peer is initiated by the peer -- they send an RST packet. Seems like a network issue perhaps, because 7 minutes earlier CIFS is "reconnecting" -- what was the cause of that? Maybe on the client see if vers=3.0 works on the mount options?

    True. kodiplayer is the server (LE11 nightly), grabber the client (DietPi).

    I thought that the issue was with the client, like you, but I ruled it out by using my home NAS as the server. All went well.

    I confirmed that it was a LE11 server issue when I fresh installed kodiplayer with Raspberry Pi OS. Same server hardware, different server software, same client hardware and software. Same network in all cases.

    I also have a copycat production setup working seamlessly. The software is exactly the same, configured exactly the same way.

    The only difference between the 2 kits is the newer hardware revision of the Pi 4 server.

    It's all pretty much pointing at a problem on the server.

    But I'm not one to ignore suggestions, I will totally test with vers=3.0 and report back!

    Quote from HiassofT

    FYI: new nightly with kernel/firmware update is now available https://test.libreelec.tv/LibreELEC-RPi4…203-d383c10.tar

    so long,

    Hias

    oh, this is what I've been running all day long, as mentioned on post #18 (RE: RPi 4 2 GB losing USB - Firmware issue?). Can you hear my optimism getting shattered? :D

    Edited once, last by camelreef: Merged a post created by camelreef into this post. (February 3, 2022 at 11:46 PM).

    Quote from camelreef

    oh, this is what I've been running all day long, as mentioned on post #18 (RE: RPi 4 2 GB losing USB - Firmware issue?).

    This is NOT the same file.

    Rev 1.5 RPi 4 - LE11 nightly-20220203-5347c5e

    LibreELEC-RPi4.arm-11.0-nightly-20220203-d383c10.tar

    There are now 2 Nightly builds dated 20220203 so you have to look at the filename tail...

    Believe to HiassofT and give it a try. `

    Edited once, last by ghtester (February 8, 2022 at 10:57 AM).

    Quote from camelreef

    I thought that the issue was with the client, like you, but I ruled it out by using my home NAS as the server. All went well.


    I confirmed that it was a LE11 server issue when I fresh installed kodiplayer with Raspberry Pi OS. Same server hardware, different server software, same client hardware and software. Same network in all cases.

    So you ran Samba on Raspberry Pi OS and did the same testing and everything was fine? Interesting. I kind of got lost in all your updates. :)

    Well, the other thing to point out is there should be a Samba bump to 4.13.17 in tomorrow's nightly along with HiassofT's updates, so you can maybe look out for that as well. Personally, I don't use Samba at all with Kodi/LE, just NFS. Samba has a lot of protocol overhead, that NFS doesn't have. Scanning videos on a large library takes a long time over the network with Samba. I also only use the Pi to play media, that's all, no docker crap, no samba server crap (it's disabled), not even cron.

    Beyond that, perhaps you need to fire up Wireshark and do some packet capturing to see what is happening at a protocol level. I assume the only problem left is this Samba issue? I am assuming the crash on kodiplayer is because the connection was reset from the peer.

    Quote from ghtester

    This is NOT the same file.

    Rev 1.5 RPi 4 - LE11 nightly-20220203-5347c5e

    LibreELEC-RPi4.arm-11.0-nightly-20220203-d383c10.tar

    Ther are now 2 Nightly builds dated 20220203 so you have to look at the filename tail...

    Believe to HiassofT and give it a try. `

    Oh, yes! Apologies offered!

    I'm so used to the clockwork daily releases that I did not consider a double release day!

    As Mulder used to say... I want to believe!

    Quote from frakkin64

    So you ran Samba on Raspberry Pi OS and did the same testing and everything was fine? Interesting. I kind of got lost in all your updates. :)

    Well, the other thing to point out is there should be a Samba bump to 4.13.17 in tomorrow's nightly along with HiassofT's updates, so you can maybe look out for that as well. Personally, I don't use Samba at all with Kodi/LE, just NFS. Samba has a lot of protocol overhead, that NFS doesn't have. Scanning videos on a large library takes a long time over the network with Samba. I also only use the Pi to play media, that's all, no docker crap, no samba server crap (it's disabled), not even cron.

    Beyond that, perhaps you need to fire up Wireshark and do some packet capturing to see what is happening at a protocol level. I assume the only problem left is this Samba issue? I am assuming the crash on kodiplayer is because the connection was reset from the peer.

    I totally agree with you about NFS vs Samba. There is no contest. I'm an old Sun Solaris hand, NFS is second nature to me. I use NFS from my NAS on my home system (see sig)!

    But the current kits I'm building for family and friends need to be compact and self sustaining. I need the HDD connected to the LE Pi for I/O perf on 4K HDR media files (it also removes the scanning pain you mention), as it only plays media. The traffic over the network share is less critical wrt to perfs, but it does have to work!

    Let's see what that new build brings to the table before pulling out the protocol analyser! I'm trusting the team!

    Edited once, last by camelreef: Merged a post created by camelreef into this post. (February 4, 2022 at 7:43 AM).

    Quote from camelreef

    But the current kits I'm building for family and friends need to be compact and self sustaining.

    Ahh, I get it. Catering to a more novice user base that is familiar with Windows. I don't envy the tech support you will have to endure.

    Quote from camelreef

    I may be jumping the gun, but... I would already have had a CIFS mount dropout by now... It appears to be working! Woohoo!

    Good to hear. Sounds like it was a combination of a few things, firmware bump to perhaps address the rev 1.5 board firmware timeouts/splats, and Samba bump/mount option changes, care to summarize what you think solved it? It may help others that come across this problem.

    Quote from frakkin64

    Ahh, I get it. Catering to a more novice user base that is familiar with Windows. I don't envy the tech support you will have to endure.

    Good to hear. Sounds like it was a combination of a few things, firmware bump to perhaps address the rev 1.5 board firmware timeouts/splats, and Samba bump/mount option changes, care to summarize what you think solved it? It may help others that come across this problem.

    Does LE11 have NFS server capabilities? I don't believe so, or at least it wasn't there the last time I've looked. If it made its way in, I'm switching to it instantly, dropping Samba!

    Tech support is light. They use Kodi and a couple of very well made web pages through a small portal Iv integrated on the non LE RPi.

    In any case, I always have my remote access for system maintenance, which takes away a lot of potential pain.

    So far, so good!

    As for what fixed it? Well...

    1. a lot has changed in one go: firmware, bootloader and Samba (I didn't see a version bump for Samba, though), at least. I don't believe that I can point at the main reason in any educated way. Guts say firmware...
    2. it's not completely fixed, there are still drops, but shorter and more spaced in time, so the apps relatively gracefully swallow them
    Quote from HiassofT

    FYI: new nightly with kernel/firmware update is now available https://test.libreelec.tv/LibreELEC-RPi4…203-d383c10.tar

    Things are much better! There is a very clear improvement But... it's not completely fixed, there are still drops, but shorter and more spaced in time, so the apps using the samba mount relatively gracefully swallow them without degrading into painful death

    Edited once, last by camelreef: Merged a post created by camelreef into this post. (February 4, 2022 at 12:52 PM).