First boot, stuck on boot screen

    I'm trying to use LibreELEC on my Minix Neo U9-H. I've flashed LibreELEC-AMLGLX.arm-11.0.3-box.img.gz onto a MicroSD card, put it into the Minix and held down the power button. It gets stuck on the boot screen with the LibreELEC logo. I've also tried removing the quiet parameter from the boot arguments to see what's going on. It looks like it keeps starting kodi.service, then stopping it again. The process goes like this:

    Started kodi.service

    Job wait-time-sync.service/start running (this runs for a few seconds)

    Stopped kodi.service

    Starting kodi.service

    Started kodi.service

    This goes on in an infinite loop. Could it be a bad MicroSD card or is this likely caused by some other issue?

    Add "ssh" to boot params so the daemon is forced to start and then you can SSH into the box to look at (and share) dmesg and Kodi logs. The box doesn't hang at the boot splash, it's simply never overwritten with Kodi so remains visible.

    It's possible for bad cards to cause problems, but since the entire OS (kernel and userspace) are essentially in two compressed files; if the card media is damaged it generally results in nothing booting (not partial boot). If you have a spare card you're welcome to try another, and also experiment with https://chewitt.libreelec.tv/testing/LibreE…80.0-box.img.gz (LE12).

    Quote from chewitt

    Add "ssh" to boot params so the daemon is forced to start and then you can SSH into the box to look at (and share) dmesg and Kodi logs. The box doesn't hang at the boot splash, it's simply never overwritten with Kodi so remains visible.

    It's possible for bad cards to cause problems, but since the entire OS (kernel and userspace) are essentially in two compressed files; if the card media is damaged it generally results in nothing booting (not partial boot). If you have a spare card you're welcome to try another, and also experiment with https://chewitt.libreelec.tv/testing/LibreE…80.0-box.img.gz (LE12).

    After adding the ssh parameter, the loop it gets stuck in changes slightly. Aside from kodi.service, it also keeps starting and stopping sshd.service (see image). I tried connecting via SSH, but I couldn't figure out how to do that exactly. I tried running the command ssh [email protected] on my laptop, as I found a page online that said to use that command, but it says it could not resolve the hostname. Am I doing it wrong or is this because of the ssh service being stopped and restarted constantly? In fact, is LibreELEC even connected to the internet at this point? I don't have it wired with ethernet, so it would have to use WiFi, but I obviously haven't been able to set that up in LibreELEC.

    I'll give the LE 12 image a shot as well.

    I tried the LE 12 image you sent. First I ran it without the ssh parameter. It got stuck in the same loop I initially described, except this time, it skipped stopped kodi.service, so it was just

    starting kodi.service

    started kodi.service

    starting kodi.service

    After adding the ssh parameter, it does the same thing. It doesn't constantly start and stop sshd.service like it did in LE 11. It even showed the kodi 21.0 splash screen for a second or 2, before returning to the LibreELEC boot screen looping between starting and started kodi.service again. Still can't connect via ssh using the command I mentioned earlier.

    The one other thing you can try is adding "textmode" to boot params. This boots the box to a text console (not Kodi) allowing you to poke around in the OS to look for problems using a USB keyboard. If the issue is still related to Kodi specifically it's not going to be seen (as Kodi isn't started) but... you might spot something.

    Otherwise I've no idea what the issue could be. I've installed AMLGX images on a couple of things using the same batch of files in my testing share earlier today and they all run without any major issues.

    To see more, we really need to see the systemd journal log and/or kodi.log to see what's causing the issue.

    Quote from chewitt

    The one other thing you can try is adding "textmode" to boot params. This boots the box to a text console (not Kodi) allowing you to poke around in the OS to look for problems using a USB keyboard. If the issue is still related to Kodi specifically it's not going to be seen (as Kodi isn't started) but... you might spot something.

    Otherwise I've no idea what the issue could be. I've installed AMLGX images on a couple of things using the same batch of files in my testing share earlier today and they all run without any major issues.

    To see more, we really need to see the systemd journal log and/or kodi.log to see what's causing the issue.

    I've now flashed LE 11 to a different (brand-new) SD card and am getting the same problem still. I did briefly see the Kodi splash screen again before it returned to the LibreELEC boot screen.

    What commands could I run in textmode to look for problems? I tried looking for the kodi.log file and found it, but cat kodi.log doesn't return anything, so it seems to be empty?

    Also, could the Android ROM installed on the box affect things in any way? I'm currently running a custom Android TV ROM. Could reverting it back to the stock firmware help?

    I tried connecting an ethernet cable to the box and adding the ssh parameter again. Now, after showing the Kodi splash screen, it returns to the LE bootscreen again, but the last thing it shows is Reached target time-sync.target. It no longer loops kodi.service.

    I still couldn't connect via SSH, but after entering textmode and manually starting the sshd service, it worked. The result of dmesg can be found in the attached file.

    Both kodi.log and kodi.old.log were empty.

    dmesg.log

    I'm not sure how time could impact on Kodi starting or SSH login which is what your comments hint at. Not least because this is a box with an RTC chip so while time might not always/initially be accurate (ahead according to the boot log; the last line shows time being corrected) it shouldn't ever be so far off it causes problems, and a single working boot (Linux or Android should keep time current.

    Can you run "journalctl | paste" and share the URL; this will mostly show the same log out but with timestamps.

    Quote from chewitt

    I'm not sure how time could impact on Kodi starting or SSH login which is what your comments hint at. Not least because this is a box with an RTC chip so while time might not always/initially be accurate (ahead according to the boot log; the last line shows time being corrected) it shouldn't ever be so far off it causes problems, and a single working boot (Linux or Android should keep time current.

    Can you run "journalctl | paste" and share the URL; this will mostly show the same log out but with timestamps.

    I tried doing it over SSH after manualy starting the sshd service again, but strangely enough, it wouldn't even let me connect that way now. It just said connection refused.

    Anyway, I ran the command via textmode, here's the URL:

    http://ix.io/4JTS

    I noticed that even after correcting the time, it still isn't correct. It's about two hours behind still.

    Code
    Feb 16 18:11:01 LibreELEC kernel: meson-vrtc c81000a8.rtc: registered as rtc1
Oct 25 09:21:39 LibreELEC kernel: rtc-hym8563 0-0051: registered as rtc0
Oct 25 09:21:39 LibreELEC kernel: rtc-hym8563 0-0051: setting system clock to 2023-10-25T09:21:39 UTC (1698225699)

    ^ the RTC chip seems to be doing its thing. Initial kernel time is taken from the glibc release date which is in the past but normally within six months-ish as LE uses a recent-ish release. The RTC chip is probed and then updates time, and then NTP (connman) is correcting time based on latest internet polled time:

    Code
    Oct 25 09:21:48 LibreELEC connmand[891]: ntp: adjust (jump): +2.225134 sec
Oct 25 09:21:51 LibreELEC systemd[1]: Finished wait-time-sync.service.
Oct 25 09:21:51 LibreELEC systemd[1]: Reached target time-sync.target.

    then a minute later crond shows this:

    Code
    Oct 25 09:22:40 LibreELEC crond[489]: time disparity of 360911 minutes detected

    ^ I was thinking this is the RTC corrected jump between Feb 16 and Oct 25, but Google says that's 251 days and 360911 minutes is 240 days, so that doesn't line up correctly.

    Code
    Oct 25 09:22:28 LibreELEC systemd[1]: sshd.service was skipped because no trigger condition checks were met.

    ^ this is also odd. The trigger conditions are either "sshd" in kernel command line (from boot params in uEnv.ini or extlinux.conf) or the file /storage/.cache/services/sshd.conf existing and that should be created on first boot.

    If you do "touch /storage/.cache/services/sshd.conf && systemctl restart sshd" does the daemon start?

    Can you also run "blkid | paste" and share the URL or output?

    Also for kicks, please use https://chewitt.libreelec.tv/testing/LibreE…80.0-box.img.gz to boot in case some random bug has been fixed since the last LE11 release. This is an LE12 nightly from my own branch.

    Quote

    ^ this is also odd. The trigger conditions are either "sshd" in kernel command line (from boot params in uEnv.ini or extlinux.conf) or the file /storage/.cache/services/sshd.conf existing and that should be created on first boot.

    Should the boot param be "ssh" or "sshd"? I've been writing it as "ssh", but I think it results in the same regardless?

    I just flashed the LE12 image again, booted it with the ssh parameter just to try that again, and after the resizing of the file system it rebooted, but now says

    Error in mount_storage: mount_common: Could not mount UUID=558bfa8e-4b76-8b00-eb96cffff7c3.

    It then gave a bunch of other errors of which I took a picture:

    After rebooting again, it does not give that problem again, but we are back to the same problem as before, where it gets stuck after Reached target time-sync.target


    Quote

    If you do "touch /storage/.cache/services/sshd.conf && systemctl restart sshd" does the daemon start?

    Yes, this allows me to connect.

    Quote

    Can you also run "blkid | paste" and share the URL or output?

    Here's the URL:

    http://ix.io/4JVr

    I was wondering if the box was finding an older /storage partition from the older install, but blkid shows none of the partitions that exist on eMMC have partition GUIDs or names that would match so that's a dead end. And I meant "ssh" before, so that's not an issue.

    You did mention running a custom OS and not the stock one, but I don't really see how that could impact on Linux since the box is still booting (vendor u-boot) and hooking it to boot LE from SD card. Both stock and current OS are likely to use the same u-boot code.

    There's a pattern where services are waiting for things on the SD card second partition (/storage) to become available which results in some messages in the journal log, and the overall speed of boot looks a little slow to me. However I'm used to looking at logs from devices that boot from faster eMMC so that's just perception, and my boot logs also show similar messages: and ultimately /storage does mount and things do continue. I can't explain why the sshd.conf file hasn't been created automatically, but perhaps related to the second-boot failure to mount the storage partition. However boot scripts should still detect its absence on third boot and create that conf; it's not a one-time/first-boot only thing.

    In short, I'm a little stumped /shrug

    I just tried flashing it to a USB flash drive, rather than MicroSD card, and this worked, weirdly enough! Booting was very slow though, due to the USB 2.0 ports on this box.

    So it must have something to do with the way it uses the SD card? Or the SD card itself, but I doubt that, as I tried 2 separate SD cards and both gave the same issue.

    Not unheard of but a possible faulty SD card reader. After you managed to boot from USB stick have you checked access to the SD card slot with an SD card inserted just for testing purposes.

    I'm not sure how to properly check this, but I inserted the SD card after booting, then went into System Info->Storage in Kodi, and it doesn't seem to show up there. Should it automatically show up there?

    Not sure if it would be relevant, but I also tried booting into Android, then inserting the SD card, and I was able to browse the SD card just fine there.

    Edit: running the command

    /usr/lib/kodi/kodi.sh

    when in textmode does make Kodi open up

    Edited once, last by Computeraar (October 28, 2023 at 10:43 AM).

    The SD card will be /dev/mmcblk0 so as long as that /dev/device exists you can test writing with "dd if=/dev/random /dev/mmcblk0 bs=1m" and the OS will write random data to the entire card. It will take quite a while (larger card size = longer time) but if there are problems with the card (or reader hardware) it'll throw errors and hang/halt/barf on the task. To state the obvious; this will trash whatever data is currently on the card :)

    Quote from chewitt

    The SD card will be /dev/mmcblk0 so as long as that /dev/device exists you can test writing with "dd if=/dev/random /dev/mmcblk0 bs=1m" and the OS will write random data to the entire card. It will take quite a while (larger card size = longer time) but if there are problems with the card (or reader hardware) it'll throw errors and hang/halt/barf on the task. To state the obvious; this will trash whatever data is currently on the card :)

    In Kodi, this is what I see under system Information -> Storage:

    To be clear, this is when booting from USB, and an empty, FAT32 formatted SD card is inserted.


    I also tried running the command you gave in textmode, which gave the following output:

    (I ran the command twice just in case). Seems like the output is just an explanation of how the command works. Does that indicate that it indeed did not find the SD card?


    (Also, sorry for the late response, I didn't have my Minix with me for a while)