Everything seems to be working OK at the moment. I enabled the epg running state on my docker as well as disabling the Sky epg collector and deleting the epg grabber channels which had been created by the Sky collector, so only the Freesat collectors are running on that.
I have left my Athlon 5350 system as it was with both enabled but have not enabled the 'running state'. I will have to post a question on the TVH Forum I think about how things work with regards to precedence, etc.
All channels on Freesat devices have OpenSky listed as epg source - none of them mention freesat as source, even though this is the predominant mux I schedule (11425H). I will experiment by removing the sky collector on my docker - but suspect I will have missing data for some channels.
On Freeview Pi box there is no item listed in the channels epg source.
I was not totally sure of the precedence of collector that should be set to keep the latest epg and if one will totally overwrite the information collected by another or just update it? If you understand what I mean? So if I have collected the 7 day epg with Freesat, then a later run collects it with Open Sky, does that Sky collection totally overwrite the Freesat collection or just fill in the gaps? I only have the Sky one enabled as well as the Freesat as I find info from some channels will be missing without it.
I've wondered about this as sometimes I get series/episode info in the TVH gui then at a later point I don't so I have suspected if one collector has this info it could be being overwritten by the one that doesn't.
Then the now/next info/EIT - should this be set at highest precedent or lowest? As I mention before I have set 2 different tuners with an alternate order of precedence. But both tuners still hold a full 7 day epg, although the one dockered on my server is not that often used for playback, more for recordings. So I can't really tell the difference being made by the order.
So you think it would be a good idea to enable the epg running state option? The current setting has epg update window set at 24 hours. Should this be left at that or altered?
On my Freeview box I have OTA settings:
45 4 * * *
15 14 * * *
45 18 * * *
The internal grabber cron is:
4 */12 * * *
and I have epg saved to disc every hour.
I always have a 7 day epg - the only problem is when things overun. A great example is when Wimbledon is on - this can cause delays, complete programme changes and even complete channel changes of programmes. I'm not sure how to get a 'dynamic' epg to cover these type of problems.
I do get good epg data for all channels on Freeview. Freesat can get a bit mixed up for some reason, with wrong channel info given for some channels at times. Also strange is some channels that have +1 versions seem to yield better data on the +1 versions than the main one, like series and episode info.
One thing I am not certain about on Freesat epg is the priorities. I have 3 enabled in order of preference --> OTA: UK Freesat (7), Open TV Sky UK (5) and OTA UK Freesat (EIT) (1) on my Athlon box. This means any information not gathered by Freesat will be filled in by the Sky, or possibly overwritten? I have mux schedulers set up to tune into the necessary transponder to get the 7 day scheduled epg.
I have set a different order on the TVH docker on my unraid server --> Freesat(EIT) (6), Freesat (5) and Sky (2). Yes I have multiple satellite cards.
The thinking here is that maybe the EIT Now/Next might update to cover programme overruns while recording. It doesn't seem to work like that though from what I have seen. I know there is epg running state in recording config, but I don't trust broadcasters to use that properly. This dates back to my experience with a VHS recorder (in the1990's) which had PDC (Programme delivery control). This was never implemented correctly and it often cut off the very beginning or end of a programme, which defeats the purpose of it.
Anyway update on the recording problem. I threw the usb stick away as it seemed to be having internal problems. Instead there is now an externally powered usb caddy with an old 2TB Samsung HD2040UI in it. I set up the swap file to be on there but I had the problem of the swap service not starting and also samba not starting. So I disabled the swap file all together. I also noticed I had an error in the advancedsettings file so corrected that.
I now have the recordings being made properly and also monitored one of them in action using htop on the secure shell - there didn't seem to be the constant increasing memory use, so it is pretty certain to me that the faulty usb stick was causing this issue as mentioned by HiassofT.
I'm really glad to have got this sorted out now as I have been baffled by it for some time. Thanks for everyone's contribution.
I have set up an external (powered) HDD to replace the usb flash stick and the latest recording had no problems. I shall set up some more to see how it goes. I will also set up a swap file on that now. The Rpi power is the official one to be used (from PiHut) and I have the TVHat case and a flirc. I have always only used the Freeview epg module, disabling the others.
As for AV hard drives - I have one in my main libreelec (freesat) box in the cabinet under the telly, which is an AM1 AMD Kabini APU and it works very well for simultaneous record and playback. The only problem with this APU is it does not play back DTS-MA, which I think is a well known shortcoming.
I don't use web playback to check recordings - I only use it to check live streams and normally as 'passed through' to the external media player. It is only the TV icon button that does not work on the Rpi4 - this works OK on my Athlon box but only with the webtv-vp8-vorbis-webm stream profile and passthrough. Not that I really use that tv button. If I want to stream any channel from the web browser I would choose the play button from config/channels and it plays via external player (either VLC or MPC-BE).
Thanks for your help everyone. I'll report back with any success or more problems.
Well I had run the fsck to fix the errors so looks like they have re-occurred. I was also testing the 2-part film recording - set one up in the pi web gui choosing the first part, no rec icon in the second part but the scheduled recording showed the correct start and end times for the 2 parts together. But only the first part recorded. Possible error as before.
I was going to ask what would cause the memory usage to increase as the recording progresses, wondering if it might be a memory leak - but perhaps it is caused by this file buffers issue you mention because of the faulty flash drive. This was a brand new flash drive when I installed it.
I might replace the usb stick with an external usb-caddied hard drive rather than another solid state. I have a few old HDD's lying around but they are not the AV type, just standard internal HDD. This pi box is only used to record occasionally, but I would still like to get to the bottom of the issue.
I have set another 2 part recording on my freesat box in the same way to see if that completes both halves. This records fine and the only issues I have with that box is it sometimes completely freezes on playback of files, usually after a skip forward or backwards (through adverts),
I recorded another programme just now (on Ch5HD 11:38 - 12:43) leaving my TV (downstairs) on - and using putty on my main PC (upstairs) ran htop to monitor. The Mem used increased as the recording went on, reaching around 1.9GB the last I looked at it.
It seemed to be recording OK in the web gui but then at the time it was due to finish recording the gui info read:
'There seems to be a problem with the live update feed from Tvheadend. Trying to reconnect...'
The web page refreshed and had to login again. This exact thing also happens if I try to play a live stream using the TV icon in the epg guide - opening the details and using play does work with external player as does the play icons from config/channels and services page.
Looking at the recorded file (only one) shows it only actually recorded for 32m45s, not the full 65 - ish minutes as listed in the completed recordings. I didn't notice any problem during the recording at this 32m point and it seemed to be recording right to the end. Here's the latest log. I can see another out of memory killing TVheadend, but this occurs a minute after the end point.
https://paste.libreelec.tv/holy-stingray.log
noggin ---> I only have OTA - Freeview set, all others are disabled. I realise the gobbledy gook txt is some encoding which I think only seems to occur on HD as it is clear on SD. Your point 2. I have discovered by my investigations into why things weren't working.
Split films on CH5 among others - I initially found last year that setting a recording only recorded the first part - so then I used to set both parts to record and would end up with one file the full length and one with just the first part. I do most of my recordings on my other LibreElec (freesat) box.
So on the pi I set the latest (2 part) recording choosing the first part only and it did record the whole thing (until the problem I am getting occurred losing the last few minutes). It is just in the client TV guide (and the webgui guide) it does not show the 'record icon' on the second part. But looking at the upcoming recordings I can see recording set correctly for the whole length. Maybe this was something that was fixed in the last year?
I had also edited the config.txt to gpu_mem=96 at the same time I added the advancedsettings.xml.
I used umount on /sda1 (my usb stick 32GB) and ran fsck - errors found and fixed, remounted and then rebooted the pi4.
With the debug mode still on the top left of the screen displays activity.
The MEM fluctuates for example 1595188/1871644, 1502444/1871644.
I'm not sure if the 1871644 (KB) is supposed to represent the whole ram capacity (which should be 2GB --> 2097152 KB) or the usable capacity after some reserve. This difference seems to be about 220 MB.
Should I really have a swap file? I thought it was a bad idea on both SDCards and usb sticks. I can also put in a larger usb stick than the current 32 GB if needed.
I do not have timeshift enabled on the pi4 - although I do have it set to ram only in the settings. I never found it to work properly on the pi4 so disabled it. Timeshift works on my athlon box (8GB Ram). I'll have a look at the other issues.
The next recording of an SD film completed OK with no errors according to the gui. It should have had a length of 1 h 44 m. The actual file was 1 h 36 m in length - only one file and it had the whole film to the end, mainly because the channel Film 4 usually has a fair bit of time between the end of one programme and then start of the next.. So I think there must have been another problem seeing as it missed 8 minutes recording time. Hopefully the forum at TVH might have some ideas when I put a post up.
These are the adjusted settings I made by the way (2GB Rpi4):
<advancedsettings>
<cache>
<buffermode>1</buffermode>
<memorysize>33554432/memorysize>
<readfactor>10</readfactor>
<chunksize>65536</chunksize>
</cache>
</advancedsettings>
I did make some adjustments to cache. The details are on my main PC but I set it as something like 96 MB cache and 96 MB video ram and I think 4 read. I can't remember the exact wording of the variables off the top of my head.
I was just checking the latest test recording and saw the same text encoding in the mediainfo file. I have seen this before with good recordings on my freesat libreelec athlon box. I just mentioned it in case it meant a possilble error. It may only happen on HD files or possibly depends on original source. Some of the mediainfo data shows things like 19.2E sat even though these are from terrestial aerial recordings, so I can only assume information is from the original source file that is being reused for different output mediums.
Anyway the latest recording (HD) should have been 34 mins (with pre and post buffers). The gui lists as 34 mins no errors. The actual file is just under 27 mins long. It plays back and the ending breaks up a little and finishes. So it seems some corruption may have occurred and stopped the recording but only one file exists - so I have to assume whatever the error is did not recover in time to continue the recording as there was only 7 mins to recover. I think there is sometimes a gap of 30-40 mins between a fail and restart.
I shall make a post on TVH Forum at the weekend and see what they advise to troublshoot.
I recorded a 1 hour program off Blaze (SD) yesterday and checking on it now shows no problems with that recording. I have set up a couple more recordings to see what happens - a 30 minute HD programme and a 100 minute SD programme.
One thing I noticed looking at the .ts files with mediainfo there is information regarding the times of programmes:
Excerpt from last successful (SD Channel) recording:
Service name : Blaze
Service type : MPEG-2 HD digital television
2024-04-03 00:00:00 UTC : en:Swamp Mysteries With Troy... / en:...Landry: Hogzilla: Troy searches for a monster threatening residents of a small Texas community. / show/game show / / 01:00:00 /
2024-04-03 01:00:00 UTC : en:Blaze is back at 5am / en: / show/game show / / 03:00:00 /
*Note All in clear text.
Compared to a bad recording made previous (HD Channel):
Service name : Channel 5 HD
Service type : MPEG-2 HD digital television
2024-03-31 01:00:00 UTC : en:ËX"ÄòR»'á¸ììÙ»J_¡ / en:- / show/game show / / 02:00:00 /
2024-03-31 03:00:00 UTC : en:«ÃÖæ³{èSð¯FËà / en:¶þ}Ý>[vTpV-m6})ùtõ=ÎðvWiè;×°9È×k6ìqó½q靈¾vì㸮ZKÎ×eƪå©xvé'ìFlxq: [à / social/political issues/economics / / 00:45:00 /
2024-03-31 03:45:00 UTC : en:Êr¨9N¾ýµÚ£©+¹à / en:qL¹lÓö{:c¡Ïx¿T£Ü#vÆþ¶=fÒ"¿³$^ñÎðvÇAëüª3ö{:c·¯¹K}$¾ªþ\/ÑÛU/ÓJ]QÞkâÀ¨o / social/political issues/economics / / 00:50:00 /
*Note the strange mixture of text. I don't know if this is some encryption (but all channels are free to air), or possibly corruption?
I'll make some adjustments and see what happens.
I will have to try TVHeadend forum as you say. I just wasn't sure if it was purely down to a TvHeadend problem or something within kodi or something to do with the usb stick. I am sure I did come across something during the past year I have spent looking at this issue to see if anyone else had it - that TVHeadend by design creates another recording after an interruption. I think this probably is the case as typically a recording I have made that has this error usually creates 2 files, for example:
Ghostbusters_ Afterlife-BBC ONE WMidHD2024-03-29
Ghostbusters_ Afterlife-BBC ONE WMidHD2024-03-29-1
The gui itself shows no errors and lists the runtime as correct , but the file size represents the second file downloading from the gui will only download the second one. In the client interface on the pi TV playback will only see the last file. It is only when I look at file manager I see all files created. Also there is always some ( often quite significant) amount of time missing between the end of the first file and the second.
The strange thing is these errors first started during the Womens Football World Cup last year which I was recording while at work. Although (I've mentioned previously) the recording device at that time was an old 2.5" mechanical disc from a laptop inside an external usb case, which later became corrupted.
Something that updated around this time may be the cause - I have no idea. libreelec itself was probably updated as well as TVHeadend plus any firmware / usb for the pi4.
Thanks for having a look at the logs anyway. When I enabled debugging - I set it as persistent and also enabled specifics - audio ,video , pvr. I wasn't sure how many of these specifics I should select.
Would there be information elsewhere that would be useful? I know there are a number of different log files in the zipped file that is saved. I have 9 of these zipped log files since 30.3.24 (older ones deleted), including 2 today and I've not even used the pi today.
I've looked at my unraid server and run in the terminal the command nfsstat and it shows server version as NFS V3. I've done some searching and came across various articles and V4 is does seem to be the default version to use as seen in the nfsmount.conf on the server [extract]:
[ NFSMount_Global_Options ]
# This statically named section defines global mount
# options that can be applied on all NFS mount.
#
# Protocol Version [3,4]
# This defines the default protocol version which will
# be used to start the negotiation with the server.
# limetech - start negotiation with v4
#Defaultvers=3
Defaultvers=4
#
# Setting this option makes it mandatory the server supports the
# given version. The mount will fail if the given version is
# not support by the server.
# limetech - support v4
# Nfsvers=4
I don't know if this file has to be altered where Nfsvers=4 to uncomment it. The majority of this conf file is commented out which I find a bit confusing.
Also my media shares are exported by SMB as well as NFS. I'm not sure if this makes any difference to the NFS version used. I've only updated to this latest version of Unraid on Sunday morning so it has had a reboot. I've also just stopped the array and restarted it to see if the NFS version changes but is stays at V3. I created a new test share and exported it as NFS only but it still seems to use V3. I'm not sure what I can do to make everything V4 unless I need to delete all NFS exports and then redo them all fresh.
I also looked around for information on the broken pipe on sound. To me this seems as if it should be related to actual sound output, but at this time (14:31) of the recording the TV and AVR where powered down from about 13:00. I'm not sure what to make of this.
The recording itself started at 12:03 to 14:08 then the second part of the film was 14:08 to 16:23 (the odd minutes will be padding to start and end). So the first error must have happened 51 mins in (first of the 5 files) so about 12.54.
Looking at the log file myself I see this:
2024-03-31 12:04:00.034 T:989 debug <general>: AddOnLog: pvr.hts: recording id:13092243, state:recording, title:Ben Hur, error:n/a2024-03-31 12:56:16.091 T:989 debug <general>: AddOnLog: pvr.hts: recording id:2099750975, state:scheduled, title:Ben Hur, error:n/aI think this entry at 12:56 must have been when I was checking on the recording in the TV Guide and noticed it was actually only recording the first part so I scheduled the second part to record as I hadn't noticed the short programme breaking up the film into 2 parts.
I've looked at the log file myself and it seems like there is a lot going on and too difficult for me to understand.
2024-03-31 14:32:11.677 T:2011 info <general>: AddOnLog: pvr.hts: Request async EPG (8 days)I'm also wondering if asynchronous epg might be an issue?
I use unraid server (current version 6.12.9) as NAS - I'll have to look into that. I assume that you mention this as you noticed as I don't see how it would relate to the recording errors.
The part where the stream is broken - related to audio somehow? My rpi4 is connected to a Denon AVR which then goes into a Panasonic OLED TV so the sound is set up as passthrough to the Denon for all the digital formats . I don't understand what the error is, unless it is either a data transmission problem from TV signal or a software component failing.
Probably obvious but when I boot up the rpi4 (or reboot) I need to have the TV on, the AVR on and set as rpi4 source so it can produce a picture - if the rpi4 was to restart without actively passing through to the TV then it would run with a blank screen as the EDID would not be detected. This happened recently when there had been a power cut while I had been at work, so had booted back up with everything else powered down - I could access the shares and TVheadend gui over LAN but the screen was blank until I pulled the power and reconnected with TV, etc on.
I have always used these ntp servers on anything I set up so after the correction of that mistyped one earlier I have left as is for now.
Well just back home and checking on the recording I set. This film was in 2 parts (about 2 hours first part) with some short programme of maybe 5 minutes in between so had to set the second part to record as well. The same problem - I have ended up with 5 files lengths 51m21s, 19m8s, 5m19s and 28m8s, 26m1s.
pastekodi gives this:
https://paste.libreelec.tv/noble-grouper.log.
I don't know if there is other information I can obtain - I will need some guidance of what to do.