General interface slowdown of LE11 nightlies on RPi4 after a few hours

I'm filing this on the bugs board, but it's really a placeholder.

So, here is the problem I am experiencing: I have noticed that the Kodi interface slows down quite dramatically after a few hours of uptime on both of my RPi 4.

Data points:

• both machines are RPi 4 4 GB
• One Rpi is dual headed HDMI, HDMI0 to 4K/HDR TV for video, HDMI1 to older receiver for audio
• The other RPi is single HDMI to an older 720p TV
• Both wired to the network
• Both running the latest LE11 nightlies as they get posted
• Both machines point to a shared MySQL DB for Kodi
• Both run the tvheadend add-on
• One runs the library watchdog service
• A restart of the Kodi service or a reboot restores normal GUI responsiveness
• It's only the GUI that gets slow. Playback of any type of video is fine
• No apparent memory leak
• On one Pi (running the watchdog addon) I've noticed that the kodi process regularly gets stuck hogging a core (a restart of the kodi service fixes that). But the other Pi exhibits nothing like that. Both experience the same GUI slow downs, so I would say that the CPU usage is unrelated. Normally CPU usage is very low, and still I get the GUI slow down after some time.
• GUI slowdowns are probably not linked to usage. They happen when the systems have been left alone and idle, with the TVs off for a long time.

I can run an hours long debug log and post it, but I'm afraid that I would be providing a massive haystack. I'm not refusing to do it, if told to take that path anyway.

Is there anything more targeted that I could monitor and log that would be more useful/efficient for you guys?

popcornmix OK... Can I do anything to help prove the theory or provide hard data beyond my vague description?

Quote from popcornmix

Run "perf top" when it's laggy. Leave for a minute to settle.

If you see "vc4_atomic_commit_tail"/"clk_request_start" appearing in list (and increasing in percentage), then you have the same issue.

Noted, I will report back.

Thanks!

Not ignoring, but the 2 machines have decided to behave since! It will come, it always does...

Ah... I don't know what to make of this! Two machines, two different results...

Quote from popcornmix

One doesn't seem to be getting symbols for functions, so hard to say what's happening.

The other shows openssl is repsonsible. That will be user code (not kernel) so is isn't the issue I was referring to.

Is that the on with the watchdog addon?

In either case I'd be tempted to (temporarily) disable that addon (and potentially others) to try to narrow it down.

I'll explore which as what addons. But no, the machine with OpenSSL doing strange things doesn't have the watchdog.

This is the list of addons. I'll work towards making the list as similar as possible on both ends and see what happens).

Right... I've been playing with the add-ons with no real positive result.

Then, after one of the nightly updates, the problem has gone.

A sign that you guys are doing a great job! Thanks!