General interface slowdown of LE11 nightlies on RPi4 after a few hours

  • I'm filing this on the bugs board, but it's really a placeholder.

    So, here is the problem I am experiencing: I have noticed that the Kodi interface slows down quite dramatically after a few hours of uptime on both of my RPi 4.

    Data points:

    • both machines are RPi 4 4 GB
    • One Rpi is dual headed HDMI, HDMI0 to 4K/HDR TV for video, HDMI1 to older receiver for audio
    • The other RPi is single HDMI to an older 720p TV
    • Both wired to the network
    • Both running the latest LE11 nightlies as they get posted
    • Both machines point to a shared MySQL DB for Kodi
    • Both run the tvheadend add-on
    • One runs the library watchdog service
    • A restart of the Kodi service or a reboot restores normal GUI responsiveness
    • It's only the GUI that gets slow. Playback of any type of video is fine
    • No apparent memory leak
    • On one Pi (running the watchdog addon) I've noticed that the kodi process regularly gets stuck hogging a core (a restart of the kodi service fixes that). But the other Pi exhibits nothing like that. Both experience the same GUI slow downs, so I would say that the CPU usage is unrelated. Normally CPU usage is very low, and still I get the GUI slow down after some time.
    • GUI slowdowns are probably not linked to usage. They happen when the systems have been left alone and idle, with the TVs off for a long time.

    I can run an hours long debug log and post it, but I'm afraid that I would be providing a massive haystack. I'm not refusing to do it, if told to take that path anyway.

    Is there anything more targeted that I could monitor and log that would be more useful/efficient for you guys?

  • Run "perf top" when it's laggy. Leave for a minute to settle.

    If you see "vc4_atomic_commit_tail"/"clk_request_start" appearing in list (and increasing in percentage), then you have the same issue.

    I don't think LE has updated Pi kernel too recently, so you'll have to wait for that to happen, and then you can test a nightly.

  • Run "perf top" when it's laggy. Leave for a minute to settle.

    If you see "vc4_atomic_commit_tail"/"clk_request_start" appearing in list (and increasing in percentage), then you have the same issue.

    Noted, I will report back.

    Thanks!

  • One doesn't seem to be getting symbols for functions, so hard to say what's happening.

    The other shows openssl is repsonsible. That will be user code (not kernel) so is isn't the issue I was referring to.

    Is that the on with the watchdog addon?

    In either case I'd be tempted to (temporarily) disable that addon (and potentially others) to try to narrow it down.

  • One doesn't seem to be getting symbols for functions, so hard to say what's happening.

    The other shows openssl is repsonsible. That will be user code (not kernel) so is isn't the issue I was referring to.

    Is that the on with the watchdog addon?

    In either case I'd be tempted to (temporarily) disable that addon (and potentially others) to try to narrow it down.

    I'll explore which as what addons. But no, the machine with OpenSSL doing strange things doesn't have the watchdog.

    This is the list of addons. I'll work towards making the list as similar as possible on both ends and see what happens).

    Salon Chambre (machine with the OpenSSL issue)
    audioencoder.kodi.builtin.aac v1.0.2 installed audioencoder.kodi.builtin.aac v1.0.2 installed
    audioencoder.kodi.builtin.wma v1.0.2 installed audioencoder.kodi.builtin.wma v1.0.2 installed
    game.controller.default v1.0.20 installed game.controller.default v1.0.20 installed
    game.controller.snes v1.0.19 installed game.controller.snes v1.0.19 installed
    inputstream.adaptive v20.1.2.1 installed inputstream.adaptive v20.1.2.1 installed
    kodi.binary.global.audioengine v1.1.1 installed kodi.binary.global.audioengine v1.1.1 installed
    kodi.binary.global.filesystem v1.1.7 installed kodi.binary.global.filesystem v1.1.7 installed
    kodi.binary.global.general v1.0.5 installed kodi.binary.global.general v1.0.5 installed
    kodi.binary.global.gui v5.15.0 installed kodi.binary.global.gui v5.15.0 installed
    kodi.binary.global.main v2.0.0 installed kodi.binary.global.main v2.0.0 installed
    kodi.binary.global.network v1.0.4 installed kodi.binary.global.network v1.0.4 installed
    kodi.binary.global.tools v1.0.4 installed kodi.binary.global.tools v1.0.4 installed
    kodi.binary.instance.audiodecoder v4.0.0 installed kodi.binary.instance.audiodecoder v4.0.0 installed
    kodi.binary.instance.audioencoder v3.0.0 installed kodi.binary.instance.audioencoder v3.0.0 installed
    kodi.binary.instance.game v3.0.0 installed kodi.binary.instance.game v3.0.0 installed
    kodi.binary.instance.imagedecoder v3.0.0 installed kodi.binary.instance.imagedecoder v3.0.0 installed
    kodi.binary.instance.inputstream v3.0.2 installed kodi.binary.instance.inputstream v3.0.2 installed
    kodi.binary.instance.peripheral v2.0.0 installed kodi.binary.instance.peripheral v2.0.0 installed
    kodi.binary.instance.pvr v8.0.1 installed kodi.binary.instance.pvr v8.0.1 installed
    kodi.binary.instance.screensaver v2.2.0 installed kodi.binary.instance.screensaver v2.2.0 installed
    kodi.binary.instance.vfs v3.0.1 installed kodi.binary.instance.vfs v3.0.1 installed
    kodi.binary.instance.videocodec v2.0.2 installed kodi.binary.instance.videocodec v2.0.2 installed
    kodi.binary.instance.visualization v3.0.0 installed kodi.binary.instance.visualization v3.0.0 installed
    kodi.resource v1.0.0 installed kodi.resource v1.0.0 installed
    metadata.album.universal v3.1.7 installed metadata.album.universal v3.1.7 installed
    metadata.artists.universal v4.3.7 installed metadata.artists.universal v4.3.7 installed
    metadata.common.allmusic.com v3.2.2 installed metadata.common.allmusic.com v3.2.2 installed
    metadata.common.fanart.tv v3.6.4 installed metadata.common.fanart.tv v3.6.4 installed
    metadata.common.imdb.com v3.2.6 installed metadata.common.imdb.com v3.2.6 installed
    metadata.common.musicbrainz.org v2.2.4 installed metadata.common.musicbrainz.org v2.2.4 installed
    metadata.common.theaudiodb.com v2.0.3 installed metadata.common.theaudiodb.com v2.0.3 installed
    metadata.common.themoviedb.org v3.2.18 installed metadata.common.themoviedb.org v3.2.18 installed
    metadata.generic.albums v1.0.14 installed metadata.generic.albums v1.0.14 installed
    metadata.generic.artists v1.0.14 installed metadata.generic.artists v1.0.14 installed
    metadata.local v1.0.1 installed metadata.local v1.0.1 installed
    metadata.themoviedb.org v5.2.6 installed metadata.themoviedb.org v5.2.6 installed
    metadata.themoviedb.org.python v1.6.2+matrix.1 installed metadata.themoviedb.org.python v1.6.2+matrix.1 installed
    metadata.tvshows.themoviedb.org v3.5.14 installed metadata.tvshows.themoviedb.org v3.5.14 installed
    metadata.tvshows.themoviedb.org.python v1.5.0 installed metadata.tvshows.themoviedb.org.python v1.5.0 installed
    peripheral.joystick v20.1.1 installed peripheral.joystick v20.1.1 installed
    plugin.video.arteplussept v1.1.1 installed
    plugin.video.francetv v2.0.1+matrix.1 installed plugin.video.francetv v2.0.1+matrix.1 installed
    plugin.video.iplayerwww v4.0.2 installed plugin.video.iplayerwww v4.0.2 installed
    plugin.video.tubed v1.0.5 installed
    plugin.video.youtube v6.8.18+matrix.1 installed plugin.video.youtube v6.8.18+matrix.1 installed
    pvr.hts v20.2.2.1 installed pvr.hts v20.2.2.1 installed
    repository.kodi.game v1.0.0 installed repository.kodi.game v1.0.0 installed
    repository.libreelec.tv v10.80.5 installed repository.libreelec.tv v10.80.5 installed
    repository.xbmc.org v3.3.1 installed repository.xbmc.org v3.3.1 installed
    resource.images.catchuptvandmore v1.0.13 installed
    (note: very legal thing for French TV channels)
    resource.images.iplayerwww v1.0.0 installed resource.images.iplayerwww v1.0.0 installed
    resource.images.weathericons.default v1.1.9 installed resource.images.weathericons.default v1.1.9 installed
    resource.language.en_gb v2.0.2 installed resource.language.en_gb v2.0.2 installed
    resource.uisounds.kodi v1.0.1 installed resource.uisounds.kodi v1.0.1 installed
    screensaver.kaster v1.3.6+matrix.1 installed screensaver.kaster v1.3.6+matrix.1 installed
    screensaver.xbmc.builtin.black v1.0.34 installed screensaver.xbmc.builtin.black v1.0.34 installed
    screensaver.xbmc.builtin.dim v1.0.66 installed screensaver.xbmc.builtin.dim v1.0.66 installed
    script.favourites v8.1.2 installed
    script.globalsearch v9.0.7 installed
    script.module.addon.signals v0.0.6+matrix.1 installed
    script.module.arrow v1.0.3.1 installed
    script.module.beautifulsoup4 v4.9.3+matrix.1 installed script.module.beautifulsoup4 v4.9.3+matrix.1 installed
    script.module.certifi v2020.12.05+matrix.1 installed script.module.certifi v2020.12.05+matrix.1 installed
    script.module.chardet v4.0.0+matrix.1 installed script.module.chardet v4.0.0+matrix.1 installed
    script.module.dateutil v2.8.2 installed script.module.dateutil v2.8.2 installed
    script.module.future v0.18.2+matrix.1 installed
    script.module.idna v2.10.0+matrix.1 installed script.module.idna v2.10.0+matrix.1 installed
    script.module.inputstreamhelper v0.5.10+matrix.1 installed script.module.inputstreamhelper v0.5.10+matrix.1 installed
    script.module.kodi-six v0.1.3.1 installed script.module.six v1.15.0+matrix.1 installed
    script.module.pathtools v0.1.2+matrix.2 installed
    script.module.pil v5.1.0 installed script.module.pil v5.1.0 installed
    script.module.pycryptodome v3.4.3 installed script.module.pycryptodome v3.4.3 installed
    script.module.pyxbmct v1.3.1+matrix.1 installed
    script.module.requests v2.25.1+matrix.1 installed script.module.requests v2.25.1+matrix.1 installed
    script.module.simpleeval v0.9.10 installed
    script.module.simpleplugin3 v3.0.6+matrix.1 installed
    script.module.six v1.15.0+matrix.1 installed
    script.module.soupsieve v2.1.0+matrix.1 installed script.module.soupsieve v2.1.0+matrix.1 installed
    script.module.tubed.api v1.0.12 installed
    script.module.typing_extensions v3.7.4.3 installed
    script.module.unidecode v1.1.1+matrix.2 installed
    script.module.urllib3 v1.26.4+matrix.1 installed script.module.urllib3 v1.26.4+matrix.1 installed
    script.module.watchdog v0.10.2+matrix.1 installed
    script.module.xbmcswift2 v19.0.7 installed
    script.module.youtube.dl v21.303.0+matrix.1 installed
    script.openweathermap.maps v1.0.6 installed
    script.skinshortcuts v2.0.1 installed
    script.xbmc.lcdproc v4.0.0 installed
    service.fanshim v1.0.0 installed
    service.lcdd v10.80.5.108 installed
    service.librarywatchdog v1.1.2 installed
    service.libreelec.settings v11.0 installed service.libreelec.settings v11.0 installed
    skin.estuary v3.0.10 installed skin.estuary v3.0.10 installed
    skin.quartz v5.19.0 installed
    virtual.rpi-tools v10.80.5.111 installed
    script.openweathermap.maps v1.0.6 installed
    weather.gismeteo v0.6.3+matrix.1 installed
    weather.multi v0.0.13 installed weather.multi v0.0.13 installed
    webinterface.default v19.x-2.4.8 installed webinterface.default v19.x-2.4.8 installed
    xbmc.addon v19.90.101 installed xbmc.addon v19.90.101 installed
    xbmc.core v0.1.0 installed xbmc.core v0.1.0 installed
    xbmc.gui v5.16.0 installed xbmc.gui v5.16.0 installed
    xbmc.json v12.10.1 installed xbmc.json v12.10.1 installed
    xbmc.metadata v2.1.0 installed xbmc.metadata v2.1.0 installed
    xbmc.python v3.0.1 installed xbmc.python v3.0.1 installed
    xbmc.webinterface v1.0.0 installed xbmc.webinterface v1.0.0 installed
  • Right... I've been playing with the add-ons with no real positive result.

    Then, after one of the nightly updates, the problem has gone.

    A sign that you guys are doing a great job! Thanks!