LE 8.0.2 hangs (or restarts) when left on PVR screen.

    I have removed the addon, and must say that I was mortified to find out that I had any kind of addon that is considered illegal, as I am actively against piracy and I hate to see the bad press that kodi receives because of these type of addons. I use kodi mainly for the fantastic PVR features which IMO blow any standalone PVR box out of the water (I think Krypton was the turning point in that respect). I am contributing to this forum to try and help identify issues and provide logs where possible to help the developers to address problems such as the one reported in this thread. I fully support your stance on the banned addons so will understand if you decide not to look any further into this issue. By providing these logs I only want to help improve the kodi experience for other users. I would be happy to provide more logs now that I've removed the questionable addon repo?

    Annoyingly, I've tracked back and the reason I installed that repo, and it was so I could install ITV Player. Very annoying that such a widely used addon only lives in banned repos. I looked at posts on the official kodi forums and other ITV Player users share the same frustration! I hope they manage to get it into an official repo in future. For now...lesson learnt...and I'll just have to update ITV Player manually in future (if it breaks) using links on the official Release page of the kodi forum.

    Annoyingly, this all now means that the logfiles I posted here earlier will probably never be looked at now, so this bug, which affects several legitimate PVR users will go unfixed....

    • Official Post

    It's not a problem at all. We will look further into that issue. I had a little talk with a team member and I will try to reproduce with a special build from him which will get more debug informations.

    If you want to test yourself:

    Index of /builds/debug/

    just update to those builds and test with debug logging enabled. After Kodi hangs/crashes, please get us a new logfile.

    Unfortunately I'm a bit busy at the moment. So I can't test before Friday. I will, but if you are faster....just do it ;)

    Quote from DaVu

    Unfortunately I'm a bit busy at the moment. So I can't test before Friday. I will, but if you are faster....just do it ;)

    Did you have chance to test???

    I've just been trying on my RPi3 using LibreELEC-RPi2.arm-9.0-Milhouse-20170911025601-#0910b-g3e7d5d8.tar

    After running for a couple of hours the machine has completely frozen, to the point that I cannot even SSH to it to grab the logfiles (not something I encountered previously with LE 8.1.1) :(

    I am not physically in the same location as the machine so cannot power cycle it... but will reboot tonight and see if there is a kodi.log.old. If so I shall post it up...

    Are there any other logs that will survive the reboot which may be of use?

    Quote from jahutchi

    Did you have chance to test???

    I've just been trying on my RPi3 using LibreELEC-RPi2.arm-9.0-Milhouse-20170911025601-#0910b-g3e7d5d8.tar

    After running for a couple of hours the machine has completely frozen, to the point that I cannot even SSH to it to grab the logfiles (not something I encountered previously with LE 8.1.1) :(

    I am not physically in the same location as the machine so cannot power cycle it... but will reboot tonight and see if there is a kodi.log.old. If so I shall post it up...

    Are there any other logs that will survive the reboot which may be of use?

    Following the reboot I can see there are two logs:

    kodi.old.log: eUKC

    .kodi_crashlog.log: VJDj

    I left the machine running overnight, but did not leave it in the pvr section of the home screen and it stayed stable.

    DaVu Do you require any more logs?

    I wonder if it's because my machine had completely frozen, to the point that I cannot even SSH to it to grab the logfiles.

    When it froze I left it several hours to see if it would recover itself, but the system remained unresponsive and I could not even SSH to it.

    I therefore had to manually power cycle it by unplugging the power cable, and then plugging it back in.

    The logfile I uploaded was what was left behind following this crash & hang.

    I have plenty of space on the /storage partition:

    /dev/mmcblk0p2 13.9G 3.5G 10.4G 25% /storage

    I'll try again to reproduce the problem, and see whether I get a full crash log.

    In addition you may configure your terminal program on a different machine with a large scroll back buffer and open two ssh sessions with running journalctl -f and tail -f /storage/.kodi/temp/kodi.log

    I've uploaded a new debug-enabled build #0928b: RPi2

    This includes a deadlock fix for PVR database accesses, the latest version of which appears to be getting positive results so it might be worth testing with this build in case it is related.

    Otherwise, I'd supsect a memory leak... you could confirm this by running "bcmstat.sh ZDAd 30" in a ssh console, then leave it running until the crash/hang occurs. If memory has been leaked then the "Memory Free" column will be close to zero, and the "Mem kB" columns (which reflect delta and accumulated ARM allocations respectively) will be consistently negative, indicating repeated memory allocations (ie. leakage - positive values are reported whenever a "free" occurs).

    Quote from milhouse

    I've uploaded a new debug-enabled build #0928b: RPi2

    This includes a deadlock fix for PVR database accesses, the latest version of which appears to be getting positive results so it might be worth testing with this build in case it is related.

    Otherwise, I'd supsect a memory leak... you could confirm this by running "bcmstat.sh ZDAd 30" in a ssh console, then leave it running until the crash/hang occurs. If memory has been leaked then the "Memory Free" column will be close to zero, and the "Mem kB" columns (which reflect delta and accumulated ARM allocations respectively) will be consistently negative, indicating repeated memory allocations (ie. leakage - positive values are reported whenever a "free" occurs).

    I installed the debug build #0928b on my RPi3.

    I think you are correct about a memory leak.... The machine ran stable for several hours with fairly consistent memory usage. At 9PM last night the memory spiked and the machine is now completely locked up.

    Here is the output of "bcmstat.sh ZDAd 30" from my SSH console: WS1Q9k6p

    As before, the machine is now completely unresponsive. I have a black screen, and cannot SSH/Samba to the machine whatsoever to see what (if any) logs have been created. I suspect that if I reboot I will have an incomplete crash log as before.

    Should I just power-off / power-on the machine, and see what logs were created? or is there anything I can do whilst the machine is in this state to get you the debug information you need?

    Yes, that's definitely a memory leak, or at least there's a process that is using far, far too much memory. The out of memory (OOM) process killer should eventually kill whatever process is consuming all of the memory (be it kodi.bin, or some other process) and then the system should return to normal however it can take some time (10s of minutes, maybe an hour or two) for the OOM to trigger depending on the amount of RAM that is left available (and it might not trigger at all if there is just enough free RAM to prevent OOM from triggering).

    If you can leave the system alone for a few hours then, assuming it comes back to life, can you upload the contents of "journalctl -a | pastebin" as this would confirm which process caused the OOM.

    Do you have any idea what process or activity is scheduled to run at 9pm? It would be interesting to know if the timing is consistent, as that might give a clue as to what process/activity is running.

    The entry at 21:00:25 is interesting, as that shows all of the GPU and ARM memory being freed which would normally only happen when kodi.bin is exited, ie. when Kodi has been shut down. So whatever is subsequently using all of the ARM RAM, it's probably NOT Kodi as that is no longer running (or maybe it has crashed, but I wouldn't expect a crashing kodi.bin to use all of the ARM RAM so that seems unlikely).

    What else is installed on your system, that would be terminating kodi.bin at 9pm? This may explain why the system appears "frozen" as Kodi is no longer running, however if the device no longer responds to ping or ssh then the system may still be struggling due to the OOM situation.

    Based on the time when you posted, and the timings in your log, I'm guessing the system has already been frozen for several hours, in which case pulling the plug is probably the only option.

    Quote from milhouse

    Do you have any idea what process or activity is scheduled to run at 9pm? It would be interesting to know if the timing is consistent, as that might give a clue as to what process/activity is running.

    As far as I'm aware I don't have anything unusual running at 9pm. This is a fairly minimal build with just a handful of add-ons. I'd be happy to wipe it clean to a vanilla install with only the pvr.hts addon installed and re-test if that would help..... since I have a backup of all my key folders (.config / .kodi etc)

    FTR: The previous crash from last Thursday occurred at around 10.25 in the morning so I'm thinking the time is a red-herring.

    When the system crashed I was also tailing kodi.log in a separate window - here are the final entries:


    I've also pasted as much content as I had from this SSH session to here:

    YijkcK50

    It's important to note that the problem only seems to occur when I leave it in the PVR section of the Home window.

    When I reproduce the issue on LibreELEC 8.0.x / 8.1.x then kodi.bin still crashes, but respawns quite happily (by the OOM handler?) and I do get a crash log (I posted this up earlier in this thread).

    I shall leave the machine in it's hung state to see if it recovers so I can run the journalctl command - though it's now been hung for 16 hours, so not holding out much hope....