PXE Boot stuck on reboot/shutdown

1st Official Post

    Following on from this thread about problems with USB for installation (LibreELEC) I went playing with PXE booting my libreelec installations. (pfsense for dhcp, solarwinds tftp, kernsafe iscsi server, vhdx images build with an ubuntu server VM).

    I used the following links to get my installation going and the machines are booting up fine.

    Network Boot - iSCSI - OpenELEC

    Manual Installation - OpenELEC

    The problem is on reboot or shutdown the system hangs at "reached target shutdown". My guess is that the network is shutting down before the unmount happens cleanly. Can anyone shed some light on how I might correct this?

    For comparison I tested with OE 6.0.3 and this has no problem booting/rebooting/shutting down etc. Let me know if there is anything you need from my side. I can upload the image files if anyone wants to reproduce the issue.

    Can anyone provide assistance with this one? I went back and tried 7.0.0 and again it gets stuck on shutdown/reboot. Was there any changes to iSCSI/PXE support in LE?

    So I've been testing this some more and it works with OE6.0.3 and its broken in OE6.95.3/LE7.0. The older version of OE is using Kernel 4.1 while the newer versions have changes to 4.4. Is it possible to revert the kernel in LE to a lower version so I can test/confirm this?

    Yes PXE is still supported. The LE 7.90.x alphas shut down the system a little differently than 7.0.x (and OE 6.90.x) which would try to unmount /flash and /storage, potentially causing problems when still in use. Shut down logs would be useful, "journalctl -a | pastebinit".

    Quote from x010


    My journalctl doesn't log the previous sessions, is there anyway i can change it so i can provide a log?


    With 7.90.007, add "debug" to the end of your kernel command line, then reboot. Reproduce a failed shutdown, then on the next boot "journalctl -a | pastebinit". You can now remove "debug".

    I've added the debug flag on my kernel command but i can't get journalctl to log the shutdown.

    Code
    kernel /libreelec/alpha/KERNEL
    append ip=dhcp boot=NFS=10.0.0.5:/srv/tftp/libreelec/alpha disk=NFS=10.0.0.5:/srv/nfs/storage/libreelec/alpha quiet splash ssh debug


    Here's the latest log JBGM

    Right sorry, add "debug debugging" - the first enables kernel debug, the second enables systemd debug (and preserves the journal across boots).

    After you've enabled the debugging, reboot - presumably your system will hang, so power cycle etc.

    Once the system is back up, run the following:

    Code
    logger rebootmarker && sync
reboot

    The first command just adds an entry to the log which will make finding where you started the shutdown that bit easier.

    Hopefully the system will hang on shutdown, in which case power cycle.

    After the system is back up, run:

    Code
    journalctl -a | pastebinit

    and paste the link. Thanks.

    Thanks. I haven't had a chance to go through your log in detail, but I suspect the LibreELEC systemd shutdown sequence needs tweaking to fully support any type of network booting.

    At the moment I believe the device (PC, RPi3 etc.) is hanging before restarting, most probably because the network (connmand) or some other vital network-related resource has been shut down too early, which is fairly bad news when booting over the network...

    Edited once, last by milhouse (October 24, 2016 at 10:01 PM).

    Hey.

    I've got the same problem. Worked on openelec, but not libreelec.
    Here's my debug from the shutdown:

    Thankfull for some help.

    Quote from milhouse


    At the moment I believe the device (PC, RPi3 etc.) is hanging before restarting, most probably because the network (connmand) or some other vital network-related resource has been shut down too early, which is fairly bad news when booting over the network...

    That is exactly the problem. But there is a solution. To circumvent connman to shutdown the interface on reboot/shutdown you need to prevent connman to manage the interface.
    Connman has an option to ignore interfaces via a blacklist in its configuration. The configuration is hard coded in squashfs image. But there is a script (/usr/lib/connman/connman-setup) that checks for a custom config before starting connman. Simply copy /etc/connman/main.conf to /storage/.config/connman_main.conf and add the network interface to NetworkInterfaceBlacklist option.

    Edited once, last by Goeran (November 19, 2016 at 7:41 PM).

    Quote from Goeran

    That is exactly the problem. But there is a solution. To circumvent connman to shutdown the interface on reboot/shutdown you need to prevent connman to manage the interface.
    Connman has an option to ignore interfaces via a blacklist in its configuration. The configuration is hard coded in squashfs image. But there is a script (/usr/lib/connman/connman-setup) that checks for a custom config before starting connman. Simply copy /etc/connman/main.conf to /storage/.config/connman_main.conf and add the network interface to NetworkInterfaceBlacklist option.

    Worked like a charm!
    Thank you