LibreElec v11 on RPI3 won't start

  • LE 9 & 10 are working just fine on this RPI3. Tried several times to get v11 to work, but it always stops on loading with the following error:

    Code
    error in mount storage: mount_common

    and then goes into a debug mode waiting for input.
    Is this a known issue?

  • Update... Looks like the error only happens on large µSD cards. On a 16gb card it loads fine, on my 256gb card I always get this error. Hopefully someone who knows more than I do can take a look and see what's up with larger cards and LE11.

  • More info...

    Installed LE10 on the 256g card. Boots and runs fine.

    Selected the "Upgrade" function in LE, and upgraded the install to LE11.

    Upon boot, here's what I see:

    Code
    *** Error in mount_sysroot: mount_common: Could not mount /flash/SYSTEM *** wit Starting debugging shell for boot step: mount_sysroot... type exit to quit ###

    Then, I see the LE Logo, and soon after:

    Code
    *** Error in mount_sysroot: mount_common: Could not mount /flash/SYSTEM ***
    ### Starting debugging shell for boot step: mount_sysroot... type exit to quit ###
    # exit
    *** Error in mount_storage: mount_common: Could not mount UUID-9936eede-ceOf-492a-b25e-953244149f4? ***
    ### Starting debugging shell for boot step: mount_storage... type exit to quit ###
    # exit
    mount: mounting /flash on /sysroot/flash failed: No such file or directory mount: mounting /storage on /sysroot/storage failed: No such file or directory
    *** Error in prepare_sysroot: final_check: Could not find systemd! ***
    ### Starting debugging shell for boot step: prepare_sysroot..
    .. type exit to quit ###

    The "exit"s are what I typed to see if I could get any further.

    Eventually it fails with a kernel panic error.

    Hopefully this info helps in figuring out what needs to be done.

  • Ok, even more troubleshooting while I'm waiting to see if anyone finds a solution...

    Taking the working 16gb card which had already expanded itself and was running LE11 with no issues, I cloned that card to my 256gb card. Put it in the RPI3 and tried to boot it up, but sadly had the same result as earlier.

    Seems LE11 is having trouble reading the boot record on larger µSD cards?

    Hopefully it's just a configuration switch to get it up and running again.

  • Thank you for your reply, petediscrete , but I'm not sure how that article is relevant?

    This same card works perfectly in everything I use it on, including this particular RPI3 when running LE9 or LE10.

    As I suggested do your own research and you’ll see the relevancy.

    Just curious why you need such a high capacity card for LE. You can off load everything else to a USB stick or other external drive.

    Maybe in the meantime someone from devs might have other suggestions.

  • Meestor_X

    Exiting the "debug shell" resulting in a panic is normal, because the kernel can't mount disks. A few things to try:

    1. Try "blkid" and report back what that shows in the shell.

    2. Try "dmesg" and see if you can capture the output in any meaningful way, looking to see if the mmc is detected. Alternatively you can try to filter it down to things that say mmc or sdhc with "dmesg|egrep -i '(mmc|sdhc)'" (without double quotes). You can also try piping dmesg output into more with "dmesg |more" if you want to capture each page on the display with a camera.

    3. Try adding " sdhci.debug_quirks2=4" to the end of /flash/cmdline.txt (without quotes, same line). This drops 1.8V signaling for UHS cards and runs them at a lower speed, it's worth a shot if only to narrow down the problem.

    Lastly, it might be a good idea to test with Raspberry Pi OS and use "sudo rpi-update 889323" to bump to the 6.1.14 kernel (should be similar/close to what LE11 is running) and reboot/test to see. If it's a problem there, then I would report it over on their Github. You can also bump around different kernel versions on Raspberry Pi OS with rpi-update to try and pin-point where it started by using the commit id's from the commit log here.

    It doesn't seem like it is a known issue, or a common issue -- but one of the developers might be in the know. I only use Raspberry Pi 4's, and only 1 of them has a 128GB SD card (and it's using Raspberry Pi OS with 6.1.15 kernel) and no problems there. The rest are all 16GB or 32GB cards.

  • Thank you for your reply and suggestions, frakkin64 .

    1. Try "blkid" and report back what that shows in the shell.

    I assume you mean run these commands from the debug shell? And that's B L K I D in lowercase, correct? If so, then nothing returns from that command.

    Quote

    2. Try "dmesg" and see if you can capture the output in any meaningful way, looking to see if the mmc is detected. Alternatively you can try to filter it down to things that say mmc or sdhc with "dmesg|egrep -i '(mmc|sdhc)'" (without double quotes). You can also try piping dmesg output into more with "dmesg |more" if you want to capture each page on the display with a camera.

    nothing comes back for sdhc but for this command:

    Code
    dmesg | grep mmc | less

    I see something that might be helpful...

    Code
    mmc0: ============================================
    mmc0: error -110 reading general info of SD ext reg
    mmc0: error -110 whilst initialising SD card
    mmc0: host does not support reading read-only switch, assuming write-enable
    mmc0: timeout waiting for hardware interrupt.
    mmc0: cmd op 48 arg Oxiff flags Oxb5 - resp 00000900 00000000 00000000 00000000, err 0
    mmc0: data blocks 1 blksz 200 - err 0
    mmc0: =========== REGISTER DUMP ===========
    Quote

    3. Try adding " sdhci.debug_quirks2=4" to the end of /flash/cmdline.txt (without quotes, same line). This drops 1.8V signaling for UHS cards and runs them at a lower speed, it's worth a shot if only to narrow down the problem.

    For this I have to mount the sd card in another computer to read, and the file is in the root folder of that sd card, i.e. /cmdline.txt, correct?

    Currently, cmdline.txt has only one line:

    Code
    boot=UUID=2802-3558 disk=UUID=39f82505-54be-4c3c-946f-10a7d42304f1 quiet

    Adding sdhci.debug_quirks2=4 changes the first error message, it now shows:

    Code
    *** Error in mount_flash: mount_common: Could not mount UUID=2802-3558 ***
    ### Starting debugging shell for boot step: mount_flash... type exit to quit ###

    However, removing that line does not change the error message, it shows this same message instead of the earlier

    Code
    *** Error in mount_sysroot: mount_common: Could not mount /flash/SYSTEM ***
    Quote

    Lastly, it might be a good idea to test with Raspberry Pi OS and use "sudo rpi-update 889323" to bump to the 6.1.14 kernel (should be similar/close to what LE11 is running) and reboot/test to see. If it's a problem there, then I would report it over on their Github. You can also bump around different kernel versions on Raspberry Pi OS with rpi-update to try and pin-point where it started by using the commit id's from the commit log here.

    I might try this if I get time, but the fact that this is a VERY repeatable error (this same sd card works fine with LE9 and LE10 but once upgraded to LE11, fails) leads me to believe there's a change somewhere between 10 and 11 that affects it's ability to read larger sd cards.

    Hopefully a dev at some point who has an RPI3 and large SD card can give this a whirl and see what they find.

    Quote

    It doesn't seem like it is a known issue, or a common issue -- but one of the developers might be in the know. I only use Raspberry Pi 4's, and only 1 of them has a 128GB SD card (and it's using Raspberry Pi OS with 6.1.15 kernel) and no problems there. The rest are all 16GB or 32GB cards.

    Understood, thank you for your help so far!

  • I might try this if I get time, but the fact that this is a VERY repeatable error (this same sd card works fine with LE9 and LE10 but once upgraded to LE11, fails) leads me to believe there's a change somewhere between 10 and 11 that affects it's ability to read larger sd cards.

    Hopefully a dev at some point who has an RPI3 and large SD card can give this a whirl and see what they find.

    Yeah, that's where the Raspberry Pi OS comes into play, if it happens there then you might get more help on the Raspberry Pi Github or Forums. I don't know what the installs look like with LE11, but the impression from the release notes is that most RPi3 users may not be upgrading due to the loss of HEVC accelerated decoding. But they will be running the before mentioned kernel on Raspberry Pi OS, and the problem is probably something in the kernel since what you provided shows the kernel is failing to detect the SD card.

  • Just thought of something...

    Is 'mmc' how linux identifies a small card, and 'sdhc' how it identifies a large card? If so, then it looks like the problem is that it's trying to read a large card as a small one, which would totally make sense.

    How would I "force" it to identify the card as an sdhc?

  • I don't know what the installs look like with LE11, but the impression from the release notes is that most RPi3 users may not be upgrading due to the loss of HEVC accelerated decoding.

    The number of RPi2/3 users on LE10 and LE11 is actually quite high, during the last year the overall userbase distribution has been quite stable at roughly 40% RPi4, 30% RPi2/3, 20% Generic/x86, 10% everything else.

    LE11 is quite new which means we have roughly 5 times as many LE10 users as LE11 users, but still this means quite a lot of LE11 RPi2/3 installations.

    I'd also recommend testing with RPiOS and current "rpi-update" kernel, this will be the same as current LE11 nightly build and if there's an actual issue with the mmc driver (which is not caused by wearing out SD card, overclock, undervoltage etc) then RPi engineers would be the ones who could analyze and fix the issue - so posting on RPi forums would be best.

    Personally I've never had such an issue on RPi2/3 (mainly using Sandisk Extreme and Extreme Pro 32GB SD cards here).

    so long,

    Hias

  • The number of RPi2/3 users on LE10 and LE11 is actually quite high, during the last year the overall userbase distribution has been quite stable at roughly 40% RPi4, 30% RPi2/3, 20% Generic/x86, 10% everything else.

    Interesting, I guess it goes to prove that the forums are really only about problems. :) Do you guys publish those stats anywhere?

  • I'd also recommend testing with RPiOS and current "rpi-update" kernel, this will be the same as current LE11 nightly build and if there's an actual issue with the mmc driver (which is not caused by wearing out SD card, overclock, undervoltage etc) then RPi engineers would be the ones who could analyze and fix the issue - so posting on RPi forums would be best.

    TY for confirming. Can I run 'sudo rpi-update' or should I use the build number as frakkin64 suggested?

  • Is 'mmc' how linux identifies a small card, and 'sdhc' how it identifies a large card? If so, then it looks like the problem is that it's trying to read a large card as a small one, which would totally make sense.

    Those are just some of the driver components that SD cards (and even eMMCs) share, generally the driver will probe the device and negotiate the appropriate voltage levels and signaling that is supported by both.

    I don't know any other tricks, but based on what your describing it would seem to be a kernel issue. You'll want one of the Raspberry Pi engineers to help you further, those guys can guide you on how to turn on debugging to perhaps gather more information.

    I just know SD cards can be finicky, and paired with driver bugs it's hard for an end user to figure out because usually the error messages are pretty generic.


    TY for confirming. Can I run 'sudo rpi-update' or should I use the build number as frakkin64 suggested?

    The latest version is 6.1.19, so you should be OK with rpi-update. The reason I mentioned the commit is yesterday 6.1.16 was the latest and had wireless breakage.

    Edited once, last by frakkin64: Merged a post created by frakkin64 into this post. (March 14, 2023 at 9:24 PM).

  • Might be time to try an alternative high capacity SD card so you can emphatically state that LE is having difficulties with them. As I mentioned above I’m running LE11 off a Sandisk 64gb with no problems. I’ve seen as cards work with some systems and not with others. Again a good search will throw up these scenarios.

    If you do want to do a low level exam of your card to check it’s integrity f3 will give a full lo level analysis of it. Of course it’s entirely up to you if you think your card is flawless and want to hold out for a working solution. https://medium.com/@drawn_stories…al-f0109bef63ea

  • Might be time to try an alternative high capacity SD card so you can emphatically state that LE is having difficulties with them.

    I would agree this is a worthwhile test, if you have a fresh SD card of the same size it would be a good idea to load LE11 and test that it also has an issue.

    Even though it works with LE 9 and LE 10, you don't really know if it's kernel related or just happens to be the possibility that the SYSTEM image lands on a bad memory cell with LE11 and doesn't with LE10 or LE9. So, no need to say "but works on LE10 and LE9", this is my rationale on why it could still be an SD card failure. Although, I don't think that is the case here, because the kernel is failing to probe the card and don't think it has even gotten this far.

    SD card failure is the most likely cause of the problem. If the another similar size SD cards works with LE11, then you would want to do a destructive read/write test on the suspect card to confirm it.

    Edited once, last by frakkin64 (March 14, 2023 at 10:02 PM).