LE 10 Nightly - Timeshift data randomly discarded

In several latest LE 10 Nightly builds I am experiencing mentioned issue while playing Live TV (DVB-T2). Currently seen on 20210312 build / RPi 4B 2 GB device.

I am using Tvheadend Server 4.2 (9.80.11.125) + Tvheadend HTSP Client (8.2.3.1), Timeshift set to use RAM only, Fit to RAM, 900MB max.RAM size.

The Timeshft data are randomly discarded and it looks it's related to error: LE tvheadend[352]: htsp: 127.0.0.1 [ | Kodi Media Center ]: Disconnected

The issue happens most often when 1920x1080 / 50 FPS stream is played & some Timeshift offset is active. Randomly, after some time (looks always when the Timeshift buffer is full), usually when I try to seek / move the offset, it jumps to 0 offset, the Timeshift data disappears and starts from 0. Sometimes the Timeshift is disabled then (despite it's configured to start automatically) and it's necessary to push stop / start or switch between channels to get it activated again.

In past, sometimes the oom killer was activated, usually killing Tvheadend service. This does not occur now but the Timeshift data are getting cleared which is a bit annoying.

Thanks for the feedback, yes as I have mentioned above, Timeshift is configured to use RAM only.

I know Tvheadend 4.2 is known to have some issues, as well with Timeshift, but the current behaviour has changed a bit so that's why I decided to report it.

I have encountered the Tvheadend 4.3 is (thanks to you) available here: Tvheadend nightly builds for LibreELEC

but so far I didn't see it in official repository so I did not upgrade yet. Now when I tried to search, I see Tvheadend Server 4.3 (Alpha) - Team LibreELEC - 9.80.11.100 so I will give it a try... Thanks again!

Quote from elliottmc

I'd be interested in your experiences before upgrading...

See my first post (related to this matter) here: RE: Tvheadend nightly builds for LE

BTW. as the Tvheadend 4.3 add-on is installed in separate folder, it looks it can coexist with previous 4.2 release. To select the active / passive add-on you can enable the add-on you wish to use and disable the other one.

I have to confirm this issue after more intensive testing through several past days and it is really a nightmare.

It does not matter which Tvheadend backend is used, the same issue happens on both 4.2 and 4.3 Alpha, quite often during seek operations but randomly. The same issue on 2 different RPi 4B devices (2 GB, 1 GB), latest several LE 10 Nightly builds (currently 20210321).

So far I was not able to reproduce it with any specific tasks that I could describe but it seems to be related to this common error message in the log:

LibreELEC tvheadend[1677]: htsp: 127.0.0.1 [ | Kodi Media Center ]: Disconnected

In the same second it's reconnected again but the Timeshift data are lost.

If I am not mistaken, this means the Tvheadend client (8.2.3.1) is losing the connection to Tvheadend backend so the itself Client could be the reason of the issue? I have tried to increase the Connection / Response timeouts from defaults 10 / 5s to 20 / 15s but it did not help.

Well, the path is a bit different but here it is:

As there's no feedback or comment from developers and it seems the Timeshift issue is more worse than better in latest LE 10 Nightly on my RPi / 2GB, I decided to try a different Timeshift configuration.

As I had no idea what's happening in memory with Timeshift data / how it works, I created a tmpfs mount and instead of RAM Only, I changed the Timeshift configuration in Tvheadend backend to use /tmp/ramdisk with 800 MB size and currently I am testing how this works... At least I can see how the buffer files are created and when are discarded.

So far it looks that Kodi is sometimes loosing a pointer in Timeshift data (and gets messed) even in this configuration but the Plus is the files are not discarded immediately when this happens. But it's too early to say that's fact - it needs more thorough testing in my environment.

OK - so now I can see how it works when Timeshift in Tvheadend backend service is configured to use files and when the Timeshift is active:

There's created folder named buffer, inside appears a subfolder with buffer files (every minute a new file is created until their total size match the Timeshift Maximum size configuration).

This subfolder's name is a number - starting from 0, as soon as I switch the channel, the folder with buffer files is deleted and a new folder with incremented number is created and a new buffer files are generated. Everything looks working fine...

Now what happens - sometimes - usually when I try to move the Timeshift pointer in Kodi (with left - right arrows on RC), usually backwards to see the part of the Live stream again:

- The Timeshift pointer in Kodi does not jump as expected and it's position is lost, then jumps to left border of Timeshift bar (which represents the Timeshift buffer recorded size)

- The Timeshift feature stops - the buffer data files stays there and it's possible to play them again but it looks at some further point are corrupted

- TheTimeshift offset can't be moved to zero anymore with Timeshift running (until the channel is switched to another or stopped / started again). But when I try to move the Timeshift pointer to right Timeshift bar's border (keeping right arrow pressed) until the Seeking offset is >= negative Timeshift offset, it switches to Live stream (but it does not record to Timeshift buffer anymore). In this state any left-right arrow push starts playing the Timeshift buffer from begin.

Don't know if anyone can ever read this and understand what I mean - better try. But it's a real nightmare, how this feature (not)work.

When I tried the same with Timeshift to RAM only, it was even worse because the buffer data were immediately discarded when the Timeshift pointer was lost so there was nothing to analyze.

I don't know how it works together, if Tvheadend backend (4.3) has a bug or if that's Kodi but it looks more likely that Kodi can't work properly with Timeshift / Timeshift buffer data.

Also it looks the critical moment is when the Timeshift buffer gets full (so a circle write must start).

Another update - at least there's an issue with Tvheadend backend - Build: 4.3-1940 ~ LibreELEC Tvh-addon v9.80.11.100 (2021-03-10T21:50:53+0100)

In configuration described above it ignores the parameter Maximum size (MB): and keeps writing buffer files until the filesystem is full or the Maximum period (mins): limit is achieved.

Even though the Maximum period (mins): parameter is set as the limit (to keep some free space on filesystem), when this limit is achieved, after some time (when moving the Timeshift pointer, perhaps the left border is a critical point), the Timeshift stops working as well as described in previous post...

It looks nobody else is interested in nor has this issue.

But I have spent some time with investigation and now I know more. Maybe it's useless but let me share it anyway:

- perhaps I should rename this thread because the Timeshift data are not discarded randomly as I thought earlier.

- the behavior depends on Tvheadend server Timeshift configuration (where the Timeshift buffer should be stored - RAM or filesystem). The better solution seems to use a storage path as it's visible what's happening and the Timeshift data can be accessed / used when necessary.

- the reason is the Timeshift buffer is overflowed (when the Timeshift buffer is already full and you pause it or move the Timeshift pointer to left border). Then the Tvheadend stops overwriting the buffer data. When unpaused, Kodi can play the buffer files until end and then gets confused, jumps outside of left Timeshift border and decrementing the Timeshift offset to nonsense values.

- Tvheadend 4.3 has a serious bug in Timeshift configuration - the Maximum size (MB): option is ignored so it may lead to unpredictable results.

- only Maximum period (mins): can be used but files are created every minute so with very variable size which depends on data stream parameters so the filesystem size can't be effectively used for data streams with lower resolution.

- Kodi does not count with any possible Timeshift data buffer overflow and this issue is not covered.

Not yet as I am not registered there so far. Also it would be nice if somebody else could confirm the same to be sure I don't make a hoax.

IMHO Kodi needs a fix as well - I believe the Timeshift pointer should never jump outside the Timeshift bar.

AFAIK this is not an issue as the signal level on some channels is not perfect so I suppose that's why there are the Continuity errors. They don't increase much in general... It was an issue in past, I was struggling with a driver configuration (Astrometa DVB-T/T2 tuner) but now it works quite good.

I believe anybody else (who is also using a Live TV & Tvheadend) should be able to reproduce the described Timeshift / Tvheadend issue.

I don't know what exactly happens when the RAM only parameter is used but on filesystem it's immediately visible if the Timeshift / Maximum size limit is currently working or not.

I am using tmpfs for the Timethift buffer this way and it's pretty usable:

mkdir /tmp/ramdisk

chmod 777 /tmp/ramdisk

mount -t tmpfs -o size=800m myramdisk /tmp/ramdisk

I could accept that Timeshift stops (stops writing buffer data) when the buffer is about to overflow... But Kodi does not handle that situation (which happens) correctly (the question is what should be done in this case but I am sure the Timeshift pointer should not jump to nonsense / left border).