LE 8.0.2 hangs (or restarts) when left on PVR screen.

  • As far as I'm aware I don't have anything unusual running at 9pm.

    Yes, it's entirely possible that the timing is random.

    This is a fairly minimal build with just a handful of add-ons. I'd be happy to wipe it clean to a vanilla install with only the pvr.hts addon installed and re-test if that would help..... since I have a backup of all my key folders (.config / .kodi etc)

    Just shutdown kodi, rename /storage/.kodi to /storage/.kodi.bak and restart LibreELEC then you will have a "clean" system with no additional addons enabled - this should be enough to test, and you can restore your original installation (if you wish) by deleting .kodi and then renaming .kodi.bak to .kodi.

    PVR HTS has been implicated in OOM crashes but a definite cause is yet to be identified.

    Seeing all the GPU memory returned strongly suggests Kodi is shutting down, or crashing. This is definitely abnormal behaviour so testing a "clean" system would be a start, and then introduce add-ons one at a time until the problem can be reproduced (the last add-on in theory being the cause of the problem).

    Perhaps in addition to running bcmstat.sh, logging running processes might yield some additional information - run the following in a second ssh window:

    Code
    while [ : ]; do ps -opid,ppid,pgid,user,nice,vsz,rss,stat,time,etime,args > /storage/ps.log; sync; sleep 30; done

    then upload /storage/ps.log after the system has frozen and been restarted.

  • @millhouse Thanks for the tips - I shall do as you suggest, and try to reproduce with a "clean" system, and gather information on the running processes, etc.

    FTR: I rebooted last night and found that once more there was an incomplete crash log left behind (.kodi_crashlog.log) - XLBA

    There was also a kodi.old.log, but that only contains the same content as found in my terminal window as posted yesterday - hMAb

    I shall keep you updated on the results of my testing with a "clean" system.

  • I have just had another crash. Here are the steps I took:

    1) Stopped kodi and removed my .kodi directory.

    2) Removed the system.d files any other systemctl services I had installed (Docker, Inadyn, OpenVPN, stunnel) - these services would fail to start anyway since they were created by the addons, and I've now removed the .kodi folder.

    3) Rebooted the machine.

    4) Placed an advancedsettings.xml in my user data folder to enable debug logging.

    5) Installed only the pvr.hts addon, and configured this to point at my tvheadend server. Note: I also enabled debug logging within the addon settings.

    6) Restarted kodi for advancedsettings.xml to take effect (systemctl restart kodi)

    7) Ran some SSH terminals with bcmstat, etc as suggested.

    8) Navigated to the PVR section of the home screen and left it alone

    Within 20 minutes kodi crashed and here is the terminal output for bcmstat:

    FCXW

    This time around it recovered from the OOM condition, and kodi has re-spawned itself :) - maybe something about one of the other addons was previously preventing this....

    Therefore, this time around I have a complete crash log :)

    FMPT

    I was running a terminal with the repeating ps command loop; this is still running and constantly overwriting ps.log so the output there has moved on so will not be much use - SjPc

    Following the crash, I have played a few PVR channels, and have now left it Idle again in the PVR section of the Home screen. I'll see whether it crashes again, and if so then I'll post up some further crash logs.

    Do let me know if there are any other files required to help debug the code.

  • I've just had another crash at 10:04.

    Again, all I did was leave the machine Idle in the PVR section of the Home screen.

    It recovered from the OOM condition and here is the terminal output from bcmstat: OTJb

    And here is the kodi crash log: CdYB

    I also managed to capture the contents of journalctl at 10:04

    (there is an hour time difference because I've not set the correct timezone since resetting to a "clean" install)

    Code
    Oct 04 11:04:09 LibreELEC kodi.sh[1411]: ERROR: Exception caught on main loop. Exiting
    Oct 04 11:04:17 LibreELEC kernel: vchiq: 0: ignoring ERROR from callback to service a000c
    Oct 04 11:04:17 LibreELEC connmand[320]: ntp: adjust (slew): +0.004130 sec
    Oct 04 11:04:17 LibreELEC kodi.sh[1411]: Segmentation fault (core dumped)

    Edited once, last by jahutchi (October 4, 2017 at 2:19 PM).

  • And another crash at 14:32 - reproduced by playing a couple of channels and then leaving it Idle in the PVR section of the Home screen.

    It recovered itself once more and here is the output from bcmstat:

    hfLh

    Here is the kodi crash log:

    EOLJ

    And here are the relevant lines from journalctl:

    faZP

    Edited once, last by jahutchi (October 4, 2017 at 3:20 PM).

  • So I don't think it's an OOM, at least not initially - what I think may be happening is that it's a regular Kodi crash (for some reason) but due to the other stuff you have running (Docker, tvheadend etc.) there isn't enough free RAM for the system to process the core dump, and consequently the system locks up. Now that you've shut down Docker etc., the core dump can be processed and Kodi restarted without the system freezing. Adding 128MB of swap may help you process the crash dump in these extreme situations when running Docker etc., as might reducing the GPU memory to 256 so that there is 128MB more free RAM for the CPU.

    So what is causing Kodi to crash? That's not entirely clear... you've posted three crashlogs:

    #1 FMPT

    #2 CdYB

    #3 EOLJ

    #2 and #3 both show the same reason for the crash, a problem with PERIPHERALS::CPeripheralAddon::ProcessEvents so this may be one for garbear.

    #1 just seems like a random memory error, possibly due to insufficient power (you're not overclocked, so we can rule that out). If you run "bcmstat.sh ZDA d 30 y" do you see any entries in the "UFT" column before, during or after Kodi crashes? If so, please post them.

  • #1 just seems like a random memory error, possibly due to insufficient power (you're not overclocked, so we can rule that out). If you run "bcmstat.sh ZDA d 30 y" do you see any entries in the "UFT" column before, during or after Kodi crashes? If so, please post them.

    I'm not over-clocked and am using a very standard RPi3 with official power supply and memory card as purchased here:

    9M9DlSB

    It's important to note that I only experience these crashes when I leave the machine Idle in the PVR section of the home screen. Therefore, I tend to simply not use the PVR section of the home screen thus side-stepping the issue (I have buttons mapped for TVGuide, TVRecordings, etc). As long as I don't use the PVR section of the home screen, then my Pi3 will run stable for days on end without any crash logs generated.

    I will run some further tests on my RPi3 early next week to generate more crash logs, and this time will run "bcmstat.sh ZDA d 30 y" and post the results as you suggest.

    I'll assume you would like me to use the same debug-enabled build #0928b as a basis for this testing... unless I hear otherwise.

    As before, I shall also remove my whole .kodi folder etc to give a "clean" build, and will only install the pvr.hts addon.

    In the meantime, I'm not sure if anyone else in this thread with the same problem is able to test your build and supply similar debug logs for purpose of comparison.....??????

    Finally, my debug crash log from LE 8.1.1 as posted earlier in this thread is still available here:

    BSCj

    => Not sure whether this also points at PERIPHERALS::CPeripheralAddon::ProcessEvents as being the cause?

  • One final thought for today. When I was running "journalctl -a" I was seeing the following 4 messages reported once per second:

    Oct 05 16:38:07 Kitchen lircd_helper[457]: lircd-0.9.4d[457]: Error: could not get file information for /dev/lirc0

    Oct 05 16:38:07 Kitchen lircd_helper[457]: lircd-0.9.4d[457]: default_init(): No such file or directory

    Oct 05 16:38:07 Kitchen lircd-0.9.4d[457]: Error: could not get file information for /dev/lirc0

    Oct 05 16:38:07 Kitchen lircd-0.9.4d[457]: default_init(): No such file or directory


    I've just checked and the same is true when I run LE v8.1.2.

    I didn't worry about these messages since my remote works just fine, but just wondering if this could be related in some way? Since the code you mention is in the area of Peripherals.

    The remote/usb dongle I'm using is the Ortek generic MCE USB such as the one here:

    8mO0Bcl

  • Your PSU should be fine, just wanted to be sure - bcmstat.sh will report any irregular events with the "y" option.

    => Not sure whether this also points at PERIPHERALS::CPeripheralAddon::ProcessEvents as being the cause?

    No, it's a different cause this time (Kodi 17 is an almost completely different code base, though).

    The lirc error shouldn't be related, but it is odd - is that with #0928b as I'm pretty sure that build is using lirc 0.10.0 not 0.9.4d. Try disabling lirc in LibreELEC Settings addon.

  • The lirc error shouldn't be related, but it is odd - is that with #0928b as I'm pretty sure that build is using lirc 0.10.0 not 0.9.4d. Try disabling lirc in LibreELEC Settings addon.


    Those errors were from LE 8.1.2, but also occurred in #0928b

    Code
    Oct 04 11:14:49 LibreELEC lircd_helper[400]: lircd-0.10.0[400]: Error: could not get file information for /dev/lirc0
    Oct 04 11:14:49 LibreELEC lircd_helper[400]: lircd-0.10.0[400]: default_init(): No such file or directory
    Oct 04 11:14:49 LibreELEC lircd-0.10.0[400]: Error: could not get file information for /dev/lirc0
    Oct 04 11:14:49 LibreELEC lircd-0.10.0[400]: default_init(): No such file or directory


    I've now disabled LIRC in the LE Settings and those messages have gone.

    jahutchi do you have any joysticks or other controller attached to the RPi? Can you post your complete dmesg from #0928b.

    I do not have any joysticks or controllers attached. The only device I have attached is the USB dongle for my Ortek MCE Remote.

    Here is the output of dmesg from #0928b

    ERIH

  • If you run "bcmstat.sh ZDA d 30 y" do you see any entries in the "UFT" column before, during or after Kodi crashes? If so, please post them.

    I've been running most of the day with kodi in the PVR section of the Home screen.

    I have just had another crash and here is the crash log:

    BQCB

    In addition, here is the output of "bcmstat.sh ZDA d 30 y"

    DKPI

    As well as disabling LIRC I have now also added a 128MB swap file as you recommended in a previous post, though with the clean install this does not seem to be required in order for the crash log to complete.

  • Thanks, no under voltage, frequency cap or thermal throttling so looks like we can rule out PSU as well.

    Your latest crash is in PERIPHERALS::CPeripheralAddon::ProcessEvents again, which is weird as you have no controllers attached.

  • Yes the swap won't be required on a clean install, but may be of benefit once you load up the system with Docker etc. Also this #0928b debug-enabled build will use more RAM than a stripped down non-debug/release build, however I believe you were experiencing freezes on non-debug builds so 128MB of swap may prove useful in the long run (although obviously we would like to get to the bottom of the crashes so that they don't happen!)

  • Hello I’m having the same problem on my RPI3 and on odroid c2 I try to see what’s going wrong and I find that after the Epg’s list be charged the ram memory increase around 400 Kb leaving around 150 of free and father a little while something make ram memory go up and the system crash. I like to know if is possible to epg list go usb or sd memory and let the ram free.