Posts by mgouin

mgouin

For what it's worth, from tvheadend web UI (http://ip:9981) in the debug log, I was able to witness a playback stopped automatically. This is what is displayed. It seems that the connection is lost and then restored. But why...

Code

2021-03-15 21:49:23.772 htsp: Got connection from 127.0.0.1
2021-03-15 21:49:23.773 htsp: 127.0.0.1: Welcomed client software: Kodi Media Center (HTSPv34)
2021-03-15 21:49:23.779 htsp: 127.0.0.1 [ Kodi Media Center ]: Identified as user ''
2021-03-15 21:49:28.782 htsp: 127.0.0.1 [  | Kodi Media Center ]: Disconnected
2021-03-15 21:49:28.782 subscription: 0028: "HTSP" unsubscribing, hostname="127.0.0.1", username="127.0.0.1", client="Kodi Media Center"
2021-03-15 21:49:36.366 htsp: 127.0.0.1 [  | Kodi Media Center ]: Disconnected
2021-03-15 21:49:36.366 htsp: 127.0.0.1 [  | Kodi Media Center ]: Write error -- Broken pipe
2021-03-15 21:49:36.366 subscription: 0027: "HTSP" unsubscribing, hostname="127.0.0.1", username="127.0.0.1", client="Kodi Media Center"
2021-03-15 21:49:43.791 htsp: Got connection from 127.0.0.1
2021-03-15 21:49:43.792 htsp: 127.0.0.1: Welcomed client software: Kodi Media Center (HTSPv34)
2021-03-15 21:49:43.793 htsp: 127.0.0.1 [ Kodi Media Center ]: Identified as user ''
2021-03-15 21:49:43.988 dvr: "Le bon docteur" on "TVA": Updated ()

Display More

Full tvheadend log here: http://sprunge.us/CqEdeV

mgouin

Hello,

I'm about to try to downgrade to a previous LibreELEC version in order to solve my issue, but before doing so, I'd love to have some experienced user input about my issue.

I started running LibreELEC 9.2.0 on a Raspberry Pi 3 Model B since a while. I use it mainly as a PVR with tvheadend running locally on the PI. My setup is as follows:

2 USB ATSC tuner connected to the Pi via a powered USB hub
1 external Hard Drive connected to the Pi via a powered USB hub
tvheadend running locally on the PI

I know that in the early versions of 9.2.X, I was able to successfully sustain 2 simultaneous recordings while watching an already recorded program.

Now at LE 9.2.6, with the exact same physical setup, in about 10-15 minutes, the playback stops when there are 2 active recordings.

I can fairly easily reproduce the problem. I don't have a lot of addons running, beside tvheadend, the epg grabber and tvheadend client.

If useful, you can have have a look at my full debug log (from boot) here: http://sprunge.us/jqKa5p

In this log, I started playing an already recorded movies at:

2021-03-02 21:39:33.994 T:1937283520 NOTICE: VideoPlayer::OpenFile: pvr://recordings/tv/active/_-fond-de-train_2016-05-29, TV%20(Radio-Canada), 20160530_020000, 1026801670.pvr

Then, the playback stopped around this line:

2021-03-02 21:52:42.965 T:1494913920 ERROR: AddOnLog: Tvheadend HTSP Client: pvr.hts - Command fileRead failed: No response received

I'm no expert, but it seems as if tvheadend stops responding?

Where should I start from there?

Thanks a lot in advance for your help!

Mathieu

mgouin · Kodi crash randomly overnight mgouin March 22, 2020 at 7:10 PM

Hey no problem! Thanks for sharing your passion

Unfortunately, I did not go this deep in the development. I simply took the pre-compiled image for raspberry pi 3 from LibreELEC official download.

I ended up looking at the source code because I was trying to analyze the crash dumps generated by kodi, especially the stack trace.

Thanks!

mgouin · Kodi crash randomly overnight mgouin March 22, 2020 at 1:37 AM

Hello,

Thanks for the reply. I'm curious, do you experience similar crashes?

On my side, I did not get much progress beside a few more tests (refer to the github xbmc issue I raised).

Regarding your suggestion, I don't understand what you want me to try. The post refers to the command line argument length, vs what I thought my issue was is an internal C++ string length.

Could you please elaborate on your idea? I'm sincerely willing to try stuff as I'm mostly wandering in the dark with this issue...

Thanks again for your help!

mgouin · Kodi crash randomly overnight mgouin March 1, 2020 at 1:33 AM

For reference, I opened the kodi issue: Kodi crashes randomly overnight · Issue #17420 · xbmc/xbmc · GitHub

mgouin · Kodi crash randomly overnight mgouin February 29, 2020 at 8:23 PM

I created the following script to get every second the CPU, Temperature & Free RAM that I execute as a nohup.

Code

#!/usr/bin/bash

logfile=all.log

echo DATE,TimeOnly,CPU,%usr,%nice,%sys,%iowait,%irq,%soft,%steal,%guest,%idle,FREE,TEMP >> $logfile

# Loop indefinitely
while ( [ 1 ] ) do
    # Date stamp
    d=$(date +"%Y-%m-%d %H:%M:%S")

    # Mem (free)
    m=$(free -w | tr -s ' ' | cut -d ' ' -f 8 | head -n 2 | tail -n 1)

    # Temp
    t=$(vcgencmd measure_temp | egrep -o '[0-9]*\.[0-9]*')

    # CPU (last because it includes the pause)
    c=$(mpstat 1 1 | grep -w all | grep -v Average | tr -s ' ' | tr ' ' ',')

    printf "%s,%s,%s,%s\n" "$d" "$c" "$m" "$t" >> $logfile
done

Display More

Then, I analyzed the data with a quick python script:

Code

import pandas as pd
import matplotlib.pyplot as plt

# Read to dataframe
df = pd.read_csv('all.log', index_col='DATE', parse_dates=['DATE'])

# Plot relevant information data
df[['%usr','%nice','%sys','%iowait','%soft','TEMP']].plot()
plt.show()

Here is a screen shot of the plot, zoomed in where the crash occured:

Imgur: The magic of the Internet

Notice how the IOWait graph line jumps around the time my last crash dump was generated (kodi_crashlog_20200229052148.log)

Is there any way to debug iowait?

If it can help in debugging, I'm using the my LibreElec Pi as a DVR recorder with tvheadend running locally on the Pi. However, when the crash occurs, no recordings were in progress at this time.

thanks

mgouin · Kodi crash randomly overnight mgouin February 29, 2020 at 5:50 PM

Hello,

Do you think it's worthwhile to open an issue at XBMC? Issues · xbmc/xbmc · GitHub

Thanks

mgouin · Kodi crash randomly overnight mgouin February 23, 2020 at 3:56 AM

Hello,

I again got a very similar crash. Let me know if you need more logs, but the crash dump was at the same place (pasted below).

I do embedded programming as a living, so I tried to see if I could find anything.

I found this:

Code

Thread 1 (Thread 0x7378a9c0 (LWP 7099)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x74da9284 in __GI_abort () at abort.c:79
#2  0x74cc0938 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6
#3  0x74cbe630 in ?? () from /usr/lib/libstdc++.so.6
#4  0x74cbe6a4 in std::terminate() () from /usr/lib/libstdc++.so.6
#5  0x74cbea0c in __cxa_throw () from /usr/lib/libstdc++.so.6
#6  0x74ce788c in std::__throw_length_error(char const*) () from /usr/lib/libstdc++.so.6
#7  0x74d02b44 in std::string::_Rep::_S_create(unsigned int, unsigned int, std::allocator<char> const&) () from /usr/lib/libstdc++.so.6
#8  0x74d03b70 in std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned int) () from /usr/lib/libstdc++.so.6
#9  0x74d03c2c in std::string::reserve(unsigned int) () from /usr/lib/libstdc++.so.6
#10 0x74d03d40 in std::string::append(std::string const&) () from /usr/lib/libstdc++.so.6
#11 0x008d4298 in ?? ()
#12 0x00669aa4 in KODI::GUILIB::GUIINFO::CGUIInfoLabel::CInfoPortion::Get() const ()
#13 0x00669c34 in KODI::GUILIB::GUIINFO::CGUIInfoLabel::CacheLabel(bool) const ()
#14 0x0066a55c in KODI::GUILIB::GUIINFO::CGUIInfoLabel::GetLabel(int, bool, std::string*) const ()
#15 0x00643a28 in CGUILabelControl::UpdateInfo(CGUIListItem const*) ()
#16 0x006391ac in CGUIControlGroup::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#17 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#18 0x00640c74 in CGUIControlGroupList::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#19 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#20 0x00640c74 in CGUIControlGroupList::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#21 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#22 0x006391cc in CGUIControlGroup::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#23 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#24 0x006391cc in CGUIControlGroup::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#25 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#26 0x006391cc in CGUIControlGroup::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#27 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#28 0x006391cc in CGUIControlGroup::Process(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#29 0x0064fb04 in CGUIControl::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#30 0x0060d8f8 in CGUIWindow::DoProcess(unsigned int, std::vector<CDirtyRegion, std::allocator<CDirtyRegion> >&) ()
#31 0x00606064 in CGUIWindowManager::Process(unsigned int) ()
#32 0x0057ab90 in CApplication::FrameMove(bool, bool) ()
#33 0x0050e1a4 in CXBApplicationEx::Run(CAppParamParser const&) ()
#34 0x006a256c in XBMC_Run ()
#35 0x002b774c in main ()

Display More

It seems as if the xbmc/GUIInfoLabel.cpp at master · xbmc/xbmc · GitHub : Get()) method is causing a too big string.

From: libstdc++-v3 Source: basic_string.tcc Source File

Code

00462     _S_create(size_type __capacity, size_type __old_capacity,
00463           const _Alloc& __alloc)
00464     {
00465       typedef basic_string<_CharT, _Traits, _Alloc> __string_type;
00466       // _GLIBCXX_RESOLVE_LIB_DEFECTS
00467       // 83.  String::npos vs. string::max_size()
00468       if (__capacity > _S_max_size)
00469     __throw_length_error(__N("basic_string::_S_create"));

Please anyone... Where do I continue from here?

Thanks again!

mgouin · Kodi crash randomly overnight mgouin February 19, 2020 at 1:50 AM

Hello,

With debug log enabled I was able to catch another nighly crash. It ran for 4 days, so bear with the long debug log...

dmesg: http://ix.io/2c68

kodi.log: http://ix.io/2c69

kodi.old.log: Kodi old log for post https://forum.libreelec.tv/thread/21384-kodi-crash-randomly-overnight-and-reboot-rpi3 · GitHub (I think I reached the paste limit)

kodi_crashlog_20200218083716.log: kodi_crashlog_20200218083716.log · GitHub

Let me know if this is sufficient.

Thanks a lot!

mgouin · Kodi crash randomly overnight mgouin February 15, 2020 at 7:34 PM

Ok, I found it. [SOLVED] Reboot and Shutdown Crash

Posting it back in case it can help someone:

Code

<advancedsettings>
  <loglevel>1</loglevel>
</advancedsettings>

That should enable debug logging but without that annoying overlay on the screen.

I'll try that and see if it helps.

mgouin · Kodi crash randomly overnight mgouin February 15, 2020 at 7:25 PM

Quote

It's always good to enable debugging so the Kodi log is more detailed. Do so, restart Kodi, and reproduce the error(s) for a new log file.

It's funny, I enabled debug log yesterday, restarted and my system ran overnight without a crash. It did not restart by itself.

However, as a related question: is there a way to keep debug log enabled, but without the annoying top left overlay displayed constantly on the screen. I think I saw that in a custom xml, but I cannot find it back.

As I cannot yet pinpoint to something specific I did to cause the numerous crash every night, I'd like to keep running in debug log to hopefully catch the issue again.

Thank you in advance !

mgouin

I know it might not be of much help, but I have great success with the following tuner, I have 2 connected through a powered USB hub:

Hauppauge | Digital TV Tuner for Xbox One

(no affiliation, I just like them )

It works very well with tvheadend also.

mgouin · Kodi crash randomly overnight mgouin February 15, 2020 at 1:58 AM

Hello,

I have LE 9.2.0 running on a RPi3.

Recently, I realized that my ~~system~~ Kodi uptime was always a few hours, instead of days. I then noticed that every night ~~the~~ ~~system reboots~~ Kodi restarts by itself during the night. Normally, at the end of the evening, I just close the TV and let the RPi3 running.

When trying to look at the kodi logs (in /storage/.kodi/temp), I found the following interesting files:

Code

-rw-r--r--    1 root     root       62.5K Jan 19 22:28 kodi_crashlog_20200119222837.log
-rw-r--r--    1 root     root       79.7K Jan 21 03:34 kodi_crashlog_20200121033352.log
-rw-r--r--    1 root     root      379.7K Jan 23 01:45 kodi_crashlog_20200123014549.log
-rw-r--r--    1 root     root       70.3K Jan 24 05:15 kodi_crashlog_20200124051550.log
-rw-r--r--    1 root     root       98.1K Jan 29 01:55 kodi_crashlog_20200129015542.log
-rw-r--r--    1 root     root       69.7K Jan 30 02:58 kodi_crashlog_20200130025802.log
-rw-r--r--    1 root     root       66.3K Feb  1 00:43 kodi_crashlog_20200201004302.log
-rw-r--r--    1 root     root      277.6K Feb 10 12:38 kodi_crashlog_20200210123826.log
-rw-r--r--    1 root     root      119.9K Feb 13 03:29 kodi_crashlog_20200213032858.log
-rw-r--r--    1 root     root       56.3K Feb 14 00:49 kodi_crashlog_20200214004853.log

If it can be useful, here are the pastebin of the last 2 crashes:

Let me know if more crash dumps would help?

I briefly looked at the crash dumps, and the cause all seems to be inside the thread 1 of Kodi. Except the last crash, the previous ones all seem the same.

Code

LibreELEC:~/.kodi/temp # grep -A 2 'Program terminated' kodi_crashlog_*.log
kodi_crashlog_20200119222837.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200119222837.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200119222837.log-[Current thread is 1 (Thread 0x737c1240 (LWP 2492))]
--
kodi_crashlog_20200121033352.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200121033352.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200121033352.log-[Current thread is 1 (Thread 0x7378d240 (LWP 4001))]
--
kodi_crashlog_20200123014549.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200123014549.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200123014549.log-[Current thread is 1 (Thread 0x7371a240 (LWP 5933))]
--
kodi_crashlog_20200124051550.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200124051550.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200124051550.log-[Current thread is 1 (Thread 0x737e79c0 (LWP 10246))]
--
kodi_crashlog_20200129015542.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200129015542.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200129015542.log-[Current thread is 1 (Thread 0x737cb9c0 (LWP 392))]
--
kodi_crashlog_20200130025802.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200130025802.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200130025802.log-[Current thread is 1 (Thread 0x7375e9c0 (LWP 5980))]
--
kodi_crashlog_20200201004302.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200201004302.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200201004302.log-[Current thread is 1 (Thread 0x737f69c0 (LWP 391))]
--
kodi_crashlog_20200210123826.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200210123826.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200210123826.log-[Current thread is 1 (Thread 0x7373e9c0 (LWP 392))]
--
kodi_crashlog_20200213032858.log:Program terminated with signal SIGABRT, Aborted.
kodi_crashlog_20200213032858.log-#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
kodi_crashlog_20200213032858.log-[Current thread is 1 (Thread 0x737789c0 (LWP 15190))]
--
kodi_crashlog_20200214004853.log:Program terminated with signal SIGSEGV, Segmentation fault.
kodi_crashlog_20200214004853.log-#0  0x76f56be0 in memcpy () from /usr/lib/libarmmem-v7l.so
kodi_crashlog_20200214004853.log-[Current thread is 1 (Thread 0x737969c0 (LWP 392))]

Display More

How do I start from here?

Thanks in advance!

mgouin

Hello, for future reference, I enabled the delay of tvheadend service from the addon setting to 5 seconds and it seems to fix the problem.

Refer to screen shot below:

https://forum.libreelec.tv/core/attachmen…-clipboard-png/

mgouin

Hello,

I'm using a Raspberry Pi 3, with LE LibreELEC-RPi2.arm-9.2.0.img. I have setup TVheadend.

When I reboot the PI, both Kodi & tvheadend web ui don't list my recordings anymore. However, they are physically still stored on a 2TB USB external hard drive (/var/media/RED_2TB).

I think this relates to the tread: TVheadend problems after new update - lose list of recordings - Raspberry Pi - OSMC Forums

Is there a way to delay TVH when starting up?

FYI, my dmesg log is here: http://ix.io/27YL

Thanks!

Posts by mgouin

LibreELEC 9.2.6 cannot sustain 2 simultaneous HD recordings & 1 playback

LibreELEC 9.2.6 cannot sustain 2 simultaneous HD recordings & 1 playback

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

Kodi crash randomly overnight

How to diagnose lockup/crash? (LibreELEC 9.2.0 (RPi4.arm), 4GB, Hauppauge 950q USB tuner, TVheadend)

Kodi crash randomly overnight

TVHeadend looses recordings following reboot

TVHeadend looses recordings following reboot