please add me
I've managed to run this problem to ground in Kodi Krypton.
The problem is caused by the Now & Next Recording Widgets at the top of the PVR Section of the Home screen.
I've worked around the issue by creating a copy of the default estuary skin, and editing Home.xml to remove those two widgets:Code
- <control type="group" id="12000">
- <include content="Visible_Right_Delayed">
- <param name="id" value="livetv"/>
- <control type="grouplist" id="12001">
- <!--Commented this out to see if it resolves bug of PVR crashing in PVR Section of Home Window-->
- <!--<control type="grouplist" id="12855">
- <visible>PVR.IsRecordingTV | PVR.HasNonRecordingTVTimer</visible>
- <control type="group">
- <include content="PVRWidget">
- <param name="icon" value="$INFO[PVR.TVNowRecordingChannelIcon]" />
- <param name="header" value="$LOCALIZE" />
- <param name="label1" value="$INFO[PVR.TVNowRecordingDateTime]" />
- <param name="label2" value="$INFO[PVR.TVNowRecordingTitle][CR][COLOR=grey]$INFO[PVR.TVNowRecordingChannel][/COLOR]" />
- <control type="group">
- <include content="PVRWidget">
- <param name="icon" value="$INFO[PVR.TVNextRecordingChannelIcon]" />
- <param name="header" value="$LOCALIZE" />
- <param name="label1" value="$INFO[PVR.TVNextRecordingDateTime]" />
- <param name="label2" value="$INFO[PVR.TVNextRecordingTitle][CR][COLOR=grey]$INFO[PVR.TVNextRecordingChannel][/COLOR]" />
I've been running a couple of weeks now with Kodi in the PVR section of the home screen.... and no crashes
I notice that LibreELEC 8.2.0 has been released, but doesn't look like the kernel bug is fixed.
Any chance of an x86 build of 8.2.0 with the old 4.8 kernel?
I occasionally get this same error and lockups on my generic X86_64 NVidia machine running LE 8.0.1 though I haven't been able to figure out a pattern.
This morning I hit the error when launching a TV channel...
It started with a few Invalid Handle errors.
07:46:34.872 T:140562470610688 ERROR: (VDPAU) Error: An invalid handle value was provided.(3) at /home/a/LibreELEC.tv/build.LibreELEC-Generic.x86_64-8.0.1/kodi-fc1619b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp:1041
07:46:34.872 T:140562470610688 ERROR: Decode - avcodec_decode_video returned failure
The TV channel seemed to play OK, but the whole thing froze when I hit exit to return to the Kodi Home screen (with video still playing in the background). At this point I see several occurrences of the following error in kodi.log
07:46:44.635 T:140562470610688 ERROR: (VDPAU) Error: An invalid pointer was provided.(4) at /home/a/LibreELEC.tv/build.LibreELEC-Generic.x86_64-8.0.1/kodi-fc1619b/xbmc/cores/VideoPlayer/DVDCodecs/Video/VDPAU.cpp:1041
I tried hitting stop a few times to come out of the video but the whole thing was locked up so I just restarted kodi via SSH -> 'systemctl restart kodi'
I've attached the kodi logfile, but unfortunately don't have a debug log. (this only happens very occasionally, I cannot reproduce at will, and I don't leave debug logging in place, since I find this causes poor performance in other areas).
However, here is a kodi logfile from my occurrence this morning.
I am planning to upgrade when v8.2.0 has been released in the hope that maybe the NVidia driver bump could resolve it.
In the meantime, I'm not sure whether anyone can comment as to where the problem may lie from the logfile, and indeed whether upgrading to 8.2.0 will fix it.
OK, let me know if any further logs etc are required to help debug the issue.
If you run "bcmstat.sh ZDA d 30 y" do you see any entries in the "UFT" column before, during or after Kodi crashes? If so, please post them.
I've been running most of the day with kodi in the PVR section of the Home screen.
I have just had another crash and here is the crash log:
In addition, here is the output of "bcmstat.sh ZDA d 30 y"
As well as disabling LIRC I have now also added a 128MB swap file as you recommended in a previous post, though with the clean install this does not seem to be required in order for the crash log to complete.
The lirc error shouldn't be related, but it is odd - is that with #0928b as I'm pretty sure that build is using lirc 0.10.0 not 0.9.4d. Try disabling lirc in LibreELEC Settings addon.
Those errors were from LE 8.1.2, but also occurred in #0928bCode
- Oct 04 11:14:49 LibreELEC lircd_helper: lircd-0.10.0: Error: could not get file information for /dev/lirc0
- Oct 04 11:14:49 LibreELEC lircd_helper: lircd-0.10.0: default_init(): No such file or directory
- Oct 04 11:14:49 LibreELEC lircd-0.10.0: Error: could not get file information for /dev/lirc0
- Oct 04 11:14:49 LibreELEC lircd-0.10.0: default_init(): No such file or directory
I've now disabled LIRC in the LE Settings and those messages have gone.
jahutchi do you have any joysticks or other controller attached to the RPi? Can you post your complete dmesg from #0928b.
I do not have any joysticks or controllers attached. The only device I have attached is the USB dongle for my Ortek MCE Remote.
Here is the output of dmesg from #0928b
One final thought for today. When I was running "journalctl -a" I was seeing the following 4 messages reported once per second:
Oct 05 16:38:07 Kitchen lircd_helper: lircd-0.9.4d: Error: could not get file information for /dev/lirc0
Oct 05 16:38:07 Kitchen lircd_helper: lircd-0.9.4d: default_init(): No such file or directory
Oct 05 16:38:07 Kitchen lircd-0.9.4d: Error: could not get file information for /dev/lirc0
Oct 05 16:38:07 Kitchen lircd-0.9.4d: default_init(): No such file or directory
I've just checked and the same is true when I run LE v8.1.2.
I didn't worry about these messages since my remote works just fine, but just wondering if this could be related in some way? Since the code you mention is in the area of Peripherals.
The remote/usb dongle I'm using is the Ortek generic MCE USB such as the one here:
#1 just seems like a random memory error, possibly due to insufficient power (you're not overclocked, so we can rule that out). If you run "bcmstat.sh ZDA d 30 y" do you see any entries in the "UFT" column before, during or after Kodi crashes? If so, please post them.
I'm not over-clocked and am using a very standard RPi3 with official power supply and memory card as purchased here:
It's important to note that I only experience these crashes when I leave the machine Idle in the PVR section of the home screen. Therefore, I tend to simply not use the PVR section of the home screen thus side-stepping the issue (I have buttons mapped for TVGuide, TVRecordings, etc). As long as I don't use the PVR section of the home screen, then my Pi3 will run stable for days on end without any crash logs generated.
I will run some further tests on my RPi3 early next week to generate more crash logs, and this time will run "bcmstat.sh ZDA d 30 y" and post the results as you suggest.
I'll assume you would like me to use the same debug-enabled build #0928b as a basis for this testing... unless I hear otherwise.
As before, I shall also remove my whole .kodi folder etc to give a "clean" build, and will only install the pvr.hts addon.
In the meantime, I'm not sure if anyone else in this thread with the same problem is able to test your build and supply similar debug logs for purpose of comparison.....??????
Finally, my debug crash log from LE 8.1.1 as posted earlier in this thread is still available here:
=> Not sure whether this also points at PERIPHERALS::CPeripheralAddon::ProcessEvents as being the cause?
I've just had another crash at 10:04.
Again, all I did was leave the machine Idle in the PVR section of the Home screen.
It recovered from the OOM condition and here is the terminal output from bcmstat: OTJb
And here is the kodi crash log: CdYB
I also managed to capture the contents of journalctl at 10:04
(there is an hour time difference because I've not set the correct timezone since resetting to a "clean" install)Code
- Oct 04 11:04:09 LibreELEC kodi.sh: ERROR: Exception caught on main loop. Exiting
- Oct 04 11:04:17 LibreELEC kernel: vchiq: 0: ignoring ERROR from callback to service a000c
- Oct 04 11:04:17 LibreELEC connmand: ntp: adjust (slew): +0.004130 sec
- Oct 04 11:04:17 LibreELEC kodi.sh: Segmentation fault (core dumped)
I have just had another crash. Here are the steps I took:
1) Stopped kodi and removed my .kodi directory.
2) Removed the system.d files any other systemctl services I had installed (Docker, Inadyn, OpenVPN, stunnel) - these services would fail to start anyway since they were created by the addons, and I've now removed the .kodi folder.
3) Rebooted the machine.
4) Placed an advancedsettings.xml in my user data folder to enable debug logging.
5) Installed only the pvr.hts addon, and configured this to point at my tvheadend server. Note: I also enabled debug logging within the addon settings.
6) Restarted kodi for advancedsettings.xml to take effect (systemctl restart kodi)
7) Ran some SSH terminals with bcmstat, etc as suggested.
Navigated to the PVR section of the home screen and left it alone
Within 20 minutes kodi crashed and here is the terminal output for bcmstat:
This time around it recovered from the OOM condition, and kodi has re-spawned itself - maybe something about one of the other addons was previously preventing this....
Therefore, this time around I have a complete crash log
I was running a terminal with the repeating ps command loop; this is still running and constantly overwriting ps.log so the output there has moved on so will not be much use - SjPc
Following the crash, I have played a few PVR channels, and have now left it Idle again in the PVR section of the Home screen. I'll see whether it crashes again, and if so then I'll post up some further crash logs.
Do let me know if there are any other files required to help debug the code.
@millhouse Thanks for the tips - I shall do as you suggest, and try to reproduce with a "clean" system, and gather information on the running processes, etc.
FTR: I rebooted last night and found that once more there was an incomplete crash log left behind (.kodi_crashlog.log) - XLBA
There was also a kodi.old.log, but that only contains the same content as found in my terminal window as posted yesterday - hMAb
I shall keep you updated on the results of my testing with a "clean" system.
Do you have any idea what process or activity is scheduled to run at 9pm? It would be interesting to know if the timing is consistent, as that might give a clue as to what process/activity is running.
As far as I'm aware I don't have anything unusual running at 9pm. This is a fairly minimal build with just a handful of add-ons. I'd be happy to wipe it clean to a vanilla install with only the pvr.hts addon installed and re-test if that would help..... since I have a backup of all my key folders (.config / .kodi etc)
FTR: The previous crash from last Thursday occurred at around 10.25 in the morning so I'm thinking the time is a red-herring.
When the system crashed I was also tailing kodi.log in a separate window - here are the final entries:Code
- 20:59:00.166 T:1592038288 DEBUG: CPVRRecordings - Update - updating recordings
- 20:59:00.230 T:1942807440 DEBUG: CDirectoryProvider[pvr://channels/tv/*?view=lastplayed]: refreshing..
- 20:59:00.231 T:1410331536 DEBUG: CPVRDirectory::GetDirectory(pvr://channels/tv/*?view=lastplayed)
- 20:59:00.236 T:1942807440 DEBUG: CDirectoryProvider[pvr://recordings/tv/active?view=flat]: refreshing..
- 20:59:00.237 T:1474294672 DEBUG: CPVRDirectory::GetDirectory(pvr://recordings/tv/active?view=flat)
- 20:59:00.313 T:1942807440 DEBUG: AddOnLog: Tvheadend HTSP Client: pvr.hts - Getting play position 0 for recording Coronation Street
- 20:59:00.314 T:1942807440 DEBUG: AddOnLog: Tvheadend HTSP Client: pvr.hts - Getting play position 0 for recording Emmerdale
- 20:59:00.315 T:1942807440 DEBUG: AddOnLog: Tvheadend HTSP Client: pvr.hts - Getting play position 0 for recording Liar
- 20:59:30.238 T:1439691664 DEBUG: Thread JobWorker 1439691664 terminating (autodelete)
- 20:59:30.244 T:1410331536 DEBUG: Thread JobWorker 1410331536 terminating (autodelete)
- 20:59:30.302 T:1474294672 DEBUG: Thread JobWorker 1474294672 terminating (autodelete)
I've also pasted as much content as I had from this SSH session to here:
It's important to note that the problem only seems to occur when I leave it in the PVR section of the Home window.
When I reproduce the issue on LibreELEC 8.0.x / 8.1.x then kodi.bin still crashes, but respawns quite happily (by the OOM handler?) and I do get a crash log (I posted this up earlier in this thread).
I shall leave the machine in it's hung state to see if it recovers so I can run the journalctl command - though it's now been hung for 16 hours, so not holding out much hope....
I've uploaded a new debug-enabled build #0928b: RPi2
This includes a deadlock fix for PVR database accesses, the latest version of which appears to be getting positive results so it might be worth testing with this build in case it is related.
Otherwise, I'd supsect a memory leak... you could confirm this by running "bcmstat.sh ZDAd 30" in a ssh console, then leave it running until the crash/hang occurs. If memory has been leaked then the "Memory Free" column will be close to zero, and the "Mem kB" columns (which reflect delta and accumulated ARM allocations respectively) will be consistently negative, indicating repeated memory allocations (ie. leakage - positive values are reported whenever a "free" occurs).
I installed the debug build #0928b on my RPi3.
I think you are correct about a memory leak.... The machine ran stable for several hours with fairly consistent memory usage. At 9PM last night the memory spiked and the machine is now completely locked up.
Here is the output of "bcmstat.sh ZDAd 30" from my SSH console: WS1Q9k6p
As before, the machine is now completely unresponsive. I have a black screen, and cannot SSH/Samba to the machine whatsoever to see what (if any) logs have been created. I suspect that if I reboot I will have an incomplete crash log as before.
Should I just power-off / power-on the machine, and see what logs were created? or is there anything I can do whilst the machine is in this state to get you the debug information you need?
I have plenty of space on the /storage partition:
/dev/mmcblk0p2 13.9G 3.5G 10.4G 25% /storage
I'll try again to reproduce the problem, and see whether I get a full crash log.
I wonder if it's because my machine had completely frozen, to the point that I cannot even SSH to it to grab the logfiles.
When it froze I left it several hours to see if it would recover itself, but the system remained unresponsive and I could not even SSH to it.
I therefore had to manually power cycle it by unplugging the power cable, and then plugging it back in.
The logfile I uploaded was what was left behind following this crash & hang.