libreelec + tvheadend - local kodi fetching picons very slowly and with curl timeouts

1st Official Post

    I have LibreELEC 7.02 with tvheadend 4.0.9 running on rpi2 for some time, all works fine I should say, however recently I added picons on tvheadend and I'm having an issue, local kodi running on the same rpi (as well as a remote kodi on other rpi) apparently haves some problem fetching picons from tvheadend, it takes very long time for them to show on the channels list and looking at kodi log apparently I could see that it tries to fetch a few ones simultaneously (pipelining) and there are some 10sec. timeouts that I guess explain the long delays to get picons...

    As I said above I have a 2nd rpi on the LAN that also connects to tvheadend on the 1st rpi and this 2nd rpi shows exactly same issue, Kodi very slow to fetch picons. However I have also Kodi on a Windows PC also connecting to tvheadend on the 1st rpi and on this one picons fetch instantaneously for all channels.

    I question to myself... is this a Kodi bug? but no problem at Kodi on the Windows PC... only at LibreELEC on my rpi's... some curl lib related problem with that pipelined access method?

    I'm attaching my kodi log. Note that I enabled debug (incl. curl debug) only at 04:27:08, then I did browse channels list on kodi and it started fetching picons... timeouts at 04:28:11...

    Any ideas please?

    In the mean time I did also used pngcrush on mines to recompress and get rid of photoshop metadata and I did also reduced them significantly. Most are actually <10KB... total 2.5MB of 251 picons... anyway no change at all. Btw, these are my picons: Zippyshare.com - nos_cabo_220x132_nologo.rar

    Honestly I don't really think it is in any way related to file size... note that I said I get problem even with local kodi running on same RPi2 as tvheadend (as well as in a remote RPi connecting to that same tvheadend) BUT no problem at all with Kodi running on Windows PC and connecting to the same tvheadend on the RPi2...

    In the mean time I did also setup a RPi3 for a friend and again re-did some few tests with this with a completely clean Libreelec... and I did even test it with tvheadend42 also running on that new RPi3... same exact problem!

    One thing that I should say is that once picons are cached on kodi (after long time waiting to fetch them all) then they allways show ok, but if I clean icons cache on tvheadend (that causes icon number to change on the /imagecache/nnn urls) then kodi will also need to fetch them all again and same delays and timeouts appens again.

    I'm really surprised that no one else reports same problem... I'm having it since OE6 at least, did clean installs etc and can always reproduce it.

    Besuz: In your case are you running Libreleec (with tvheadend) on a RPi? (I ask it as I can't reproduce it on kodi running on Windows then it may be either RPi or Linux specific, not sure) and in your tvheadend in Configuration -> General -> picons path, you have it set as file:///storage/folder_with_your_picons... or and http url to a 3rd party http server with the picons?

    Thanks for your reply.

    Edited once, last by nars (August 19, 2016 at 10:34 PM).

    • Official Post

    Things you should know:
    1. there is a lot trouble involved if you use imagecache pls don't use it for local stuff (deactivate and clean)
    2. Tvheadend 4.0 has a few bugs at this area

    The simplest way to make it work:
    - install Tvh 4.2 from addons (update from 4.0->4.2 will not work)
    - copy your picons to the networkshare /Picons (snp to the vdr, srp to the tvh folder), reboot, ready

    If you are unsure what you need downloads -> srp-full.220x132-220x132.light.on.transparent_DATE.hardlink.tar.xz or/and snp-full.220x132-220x132.light.on.transparent_DATE.hardlink.tar.xz (you could use both at the same time)

    The Picons downloader is not yet implemented due major bugs in Kodi add-on system which prevent this :/

    Thanks for your reply CvH, concerning your points:

    1. I have "Image Cache" option disabled in tvheadend options (never enabled it in fact), however aparently imagecache is used anyway for local picons (i.e. when "Picons path" set to file://...), if you enable kodi debug you should see what I mean (i.e. requests to /imagecache/nnn urls as in my log).

    2. I did tried tvheadend42 with a clean config and could reproduce same exact problem.

    My picons are very similar to the ones you pointed on srp files, transparent png's, 220x132, same file names format... and yes I did put them at /storage/picons/vdr/ for tvh42 as it is set on it as default path...

    However after you mention imagecache problems I did a few tests more and instead of setting "Picons path" to file://... for local picons I did instead set it to an http:// url pointing to an apache server I have running at another pc at my LAN, and hosted picons on it for sure... the result is that now kodi at RPi fetches them very fast from the apache server! (as it should, and similar to what kodi on Windows always did)

    Also I can tell that yet before trying to host picons at apache I did also compare the debug log (with curl debug enabled) from both kodi running at RPi (with problem) and the log from kodi running at Windows (without problem)... and I could notice that there are apparently differences on the curl way to work... If I can understand corrently for kodi for RPi curl seems to use a pipelining method (to fetch picons faster? but that doesn't happen for sure, it sort of "chokes" instead) while kodi for Windows apparently doesn't do that... that makes me think that CvH may be correct, it may be all down to a bug at tvheadend imagecache code, that is triggered by these request methods from kodi on RPi... just don't know why kodi for Windows is different, but apparently it is, looking at the log... however I may be wrong and it may be just a bug at kodi/curl that doesn't show up when fetching from apache for some reason...

    For now as workaround I will just keep using picons hosted at apache and all is perfect

    Edited once, last by nars (August 21, 2016 at 8:40 AM).

    Yes, sure! that is the default for tvheadend42... and I used similar path before on 4.0.x also.

    If you check, imagecache mechanism seems always used when using these local file:///... paths.

    Nothing is logged at tvheadend when the problem happens, even with debug subsystems set to +all, anyway I did exactly as you asked, incl reset icons for all channels, then reproduced the problem (i.e. browsing the channels list) and I'm attaching both tvheadend and kodi logs and some screens.

    Btw, I notice on your screen that you use channel icons (channel name based) and not picons exactly... maybe the issue is only with picons? I notice that picons show as picon:// and channel icons show as file:/// in the user icon column in channels list, despite both set as file:///... paths as you can see on config screens, and yes I did reset them.

    • Official Post

    possible ... btw could you post the filename of a picon ? btw for what provider you use them ?
    Also could you try the snp picons, maybe they work snp-full.220x132-220x132.light.on.transparent_2016-08-19--20-24-49.hardlink.tar.xz (place them in the picons/tvh folder and untick the "prefer picons over channel name" from the settings (from the first picture you posted) after that "reset icons"

    You have all my picons (srp) at my post #5 in this topic.

    It's hard for me to try channel icons (snp) as I have no appropriate icons ready with these filenames for my channels... I did check and the ones from your url are for very very different channels :(

    Ok, I made a small script to create a new set of channel name based icon files (snp) and I can reproduce same exact issue, FYI they also use imagecache/nnn urls (despite image cache is disabled and path is set as 'file:///storage/picons/tvh/%C.png') and the delays/timeouts when kodi on RPi is fetching them are exactly same as for picons (srp).

    Btw I'm curious why I can't see anything on tvheadend log when it is serving icons even with debug enabled and subsystems set to +all... you have any idea? maybe trace debug would show something, but if I understand it needs to be enabled at tvheadend compile, right? If you have any idea I'm ready to do any tests you need.

    Edited once, last by nars (August 23, 2016 at 9:12 AM).

    • Official Post
    Quote from nars


    I can reproduce same exact issue

    very strang - this was just to be sure that it is nothing buggy with that picons

    btw are the icons working normal if you click at tvh at the epg ?

    Also I see you use 32bit png - maybe this is the problem ? (I uploaded the pictures converted to 8 bit)
    8bit.rar this would explain why it works at windows and not art rpi (maybe)

    btw you need the latest build to have the debug feature build in in tvh build.mycvh.de - activate debug in addon settings and then deactivate/activate the addon to make it work
    logfile at userdata/addons/service.tvheadend42/debug.log

    1. icons work fine on tvh web based epg, and I can tell you that I did already even right clicked image on one of them, clicked 'view image' (to get just the image on the browser) and then hit ctrl+f5 a few times fast to test browser fetching it from tvh, no problems at all in any requests, no delays...

    2. back to srp picons (config changed, reset all icons, clean image cache...), I did tried your 8bit ones and exactly same delays/timeouts fetching them on kodi on RPi.

    I really think we are on the wrong way suspecting about any issue with the image files... again note that the icons always end showing up... there are just curl timeouts (in kodi log) for kodi on RPi fetching them from tvh acting as http server... then kodi (or curl itself) apparently retries the request and eventually succeeds, just very slow waiting for these 10sec timeouts a few times to get all icons...

    I will soon try to enable tvh trace and will post if I find something...

    Edited once, last by nars (August 24, 2016 at 12:10 AM).

    Just want to add that I did found this is not RPi specific at all. Today I decided to try booting a x86 PC with libreelec 7.0.2 and test it, started with a fully clean config, installed TVH PVR Client plugin, setup it to connect to TVH on my RPi and could reproduce the problem. Then I decided to install TVH also on that new x86 libreelec (thus not depending on RPi or anything else), configured it with my channels and picons... changed TVH PVR Client plugin to connect to local TVH (127.0.0.1)... reboot... tried to browse the channels list on kodi and... I can reproduce the problem!!! I'm attaching the log.

    Btw, I did found one thing that may be related to all this: TVH is replying whole file contents to HEAD requests to imagecache urls (like if it was a GET request), and I notice on logs that kodi does a few HEAD requests... and curl is trying to reuse connections... maybe that puzzles curl (as no contents should be expected) and causes curl to "stall" connections?
    Anyway on kodi for windows curl handles this non-standard thing, and alerts it on log like this "Excess found in a non pipelined read: excess = 4540 url = /imagecache/1115 (zero-length body)", on libreelec version I see no such lines... different curl version doing difference?


    CvH: Sorry to persist, but if you try to clean image cache on tvh, then reboot libreelec (or stop/start tv service on kodi...), then when you browse channels list for 1st time on kodi in libreelec don't you see icons fetching slowly? (just scroll down the list as they apparently fetch as you scroll it) and check if you have lines like these showing on kodi log:

    Code
    00:59:51 T:140085617993472   ERROR: CCurlFile::Stat - Failed: Timeout was reached(28) for http://127.0.0.1:9981/imagecache/703
00:59:55 T:140085626386176   ERROR: CCurlFile::Stat - Failed: Timeout was reached(28) for http://127.0.0.1:9981/imagecache/799
00:59:56 T:140085374736128   ERROR: CCurlFile::Stat - Failed: Timeout was reached(28) for http://127.0.0.1:9981/imagecache/702
01:00:34 T:140085626386176   ERROR: CCurlFile::Stat - Failed: Timeout was reached(28) for http://127.0.0.1:9981/imagecache/732
01:00:34 T:140085366343424   ERROR: CCurlFile::Stat - Failed: Timeout was reached(28) for http://127.0.0.1:9981/imagecache/791


    (no need to enable debug to see just these)

    Note that it only happens for 1st time kodi is fetching them (and cleaning tvh image cache should force that, as it assignes them different numbers for their imagecache/nnn urls, and the kodi restart I mention is needed after that to get new urls from tvh). After icons fetched then kodi caches them and it will be always fast, until icons change again...

    (there is another way to force kodi re-fetching icons without cleaning image cache on tvh, by just cleaning kodi cache itself, just deleting .kodi/userdata/Thumbnails/ dir and .kodi/userdata/Database/Textures13.db file, restart kodi, then browse channels list and it will fetch icons while scroll)

    If you can't reproduce this... then I guess maybe I should search for paranormal help as I can always reproduce it... on two RPi's, now on x86, multiple clean configs... only on kodi on Windows as TVH client I can't see the problem (but from curl debug log I think there may exist differences on the way curl does requests) or using apache to serve icons then also no problem at all on any kodi client as I said before.