Advance settings for expanding cache not working..

1st Official Post

    I've added this code to my advancedsettings.xml file:

    I've looked at the Wiki post from Kodi however it looks like I've everything right but, the settings are not working. My internet is not the problem as I have 80-100 Mbps down TO my Pi. The links are not the problem either after on Jarvis the same links worked fine. Is there anytime you guys can suggest me to fix with my xml file or is there a bug with the beta software?

    I'm running 7.95.3 on the Pi 2. I'm using a 16 GB Class 10 SD Card.

    Edited once, last by dhrumil2504 (February 12, 2017 at 6:16 PM).

    • Official Post

    Perhaps there is a bug in your internet video streaming service...?
    If your internet connection is that fine, why do you need a bigger cache?
    Raspberry Pi builds have been solid for quite some time now.
    I still use the default settings with a slower internet connection than yours.

    How about uploading your kodi.log file so we can see what is going on.

    Quote from Klojum


    Perhaps there is a bug in your internet video streaming service...?
    If your internet connection is that fine, why do you need a bigger cache?
    Raspberry Pi builds have been solid for quite some time now.
    I still use the default settings with a slower internet connection than yours.

    How about uploading your kodi.log file so we can see what is going on.

    I'll try to upload the log as soon as can but, any service.. even YouTube will keep buffering at 1080/720 without the advanced settings.
    [hr]

    Quote from Iridium


    What version of LE are you running and on what device?
    Quote "I've added this code to my advancesettings.xml file:" Maybe it's just a typo but check the file is "advancedsettings.xml" (Note the "d")

    Yeh sorry about the typo.. I'm running 7.95.3 on a Pi 2. Jarvis worked great with the caching settings..

    Edited once, last by dhrumil2504 (February 12, 2017 at 6:15 PM).

    • Official Post

    Personally I wouldn't use <memorysize>0</memorysize> as all buffering is going to the SD card and not memory (This is slow compared to memory and increases the I/O on the card - which are not fast at the best of times.

    Try <memorysize>139460608</memorysize>, <readfactor>10</readfactor> (Increasing to 20 if all works)

    Quote from Iridium


    Personally I wouldn't use <memorysize>0</memorysize> as all buffering is going to the SD card and not memory (This is slow compared to memory and increases the I/O on the card - which are not fast at the best of times.

    Try <memorysize>139460608</memorysize>, <readfactor>10</readfactor> (Increasing to 20 if all works)

    It had no effect what so ever.. Kodi used 15% of RAM with my setting, 15% of RAM with yours and I even changed it to 32768000 and it STILL only used 15% of RAM. At this point I think it's either a bug with Libreelec or I need to reinstall from scratch. I'll be reinstalling later today to see what happens..

    Edited once, last by dhrumil2504 (February 12, 2017 at 7:51 PM).

    • Official Post

    Interesting, I use this on IMX6 with 2G memory and didn't see any change in memory usage either.

    <advancedsettings>
    <cache>
    <buffermode>1</buffermode>
    <memorysize>209190912</memorysize>
    <readfactor>20</readfactor>
    </cache>
    </advancedsettings>

    The readfactor does work, as you can see a change in the amount of buffering in the video playback position. readfactor=60 did break it though!

    I also created advancedsettings file and tried increasing memorysize and readfactor with buffermode 1. But it does not influence the memory consumption and it does not improve "Lags" occurring with streaming big files (over 30G) accessed over Samba via Ethernet :(


    Quote from antonsn


    I also created advancedsettings file and tried increasing memorysize and readfactor with buffermode 1. But it does not influence the memory consumption and it does not improve "Lags" occurring with streaming big files (over 30G) accessed over Samba via Ethernet :(

    Yeh, I'm also accessing large files, over the internet, the bandwidth is there but Kodi has to cache ahead or it will not playback in realtime. No matter what I do there is no change in the amount of RAM or Disk usage. I think it's bug with libreelec. I'm going to try the advancedsettings on a windows install now just to see if it's not Kodi itself.

    Edited once, last by dhrumil2504 (February 13, 2017 at 4:17 PM).

    • Official Post

    I'm not convinced it's a bug. The memory is being used, just not ALL at once. The amount of memory used, is going to be based on the size of data required for playback multiplied by the readfactor. (unsure where the 3 times memorysize comes into it).

    I use a Cubox (orig) running Archlinux as a NFS server with a Gigabit network, going to wifi on a 300Mbit extender back to Gigabit into a Cubox-i4 running LE. I've never had any issues with buffering, even with very large files.

    You need to check the usage on all sides. Remember a RPi only has a 10/100base ethernet and max speeds are only in the region of ~40Mbs.

    Use the tools available in the LE Repository, Network and System Tools. Use something like nmon/iperf to monitor the amount of data coming in and determine the maximum size received.

    The way I think of it, is as a water supply.
    [R1] ---P1--- [R2] --> U1

    R1 is a reservoir (your NAS/Internet streaming provider)
    P1 is a pipe (The bigger the pipe the more water can be supplied per second) (Your network).
    R2 is a reservoir (Kodi buffering) - The size of the "reservoir" is set by memorysize.
    The amount of water fed into the reservoir is determined by the amount being used by U1 multiplied by the readfactor.

    So as long as enough water is coming through P1 to satisfy U1 then you should have no problems. R2 will fill as quickly as possible but if P1 or R1 cannot supply enough, then lagging and buffering will occur.

    I have a feeling that if P1 can supply more to U1 than R2 can accommodate, then problems will occur (Dam burst), which might explain my problems with readfactor of 60.

    Kodi buffering will NOT help a bad network/server, nor will it help a slow processor (Wait times loading the data).

    Personally, I think the default LE settings i,e NO advancedsettings.xml should work for most users. If you must use it, adjust it incrementally and check your data.

    I'm also suspicious that my RAM usage does not appear to change regardless of what these are set to. I'm using SMB over ethernet, running 7.95.beta6 on an S905X box and am also having buffering issues. I just posted about the issue here.

    Quote from lrusak


    When you change advancedsettings.xml do you reboot each time? Or at least restart kodi to reload the new settings?

    Yes, I reboot the system.
    [hr]

    Just to check, I ran iperf3 and go pretty good speeds and I have 300 Mbps link so it's not the network..

    I'll continue messing around with readfactor but is there anything else you suggest for me to do? I know it's not the server or the Pi because it works perfectly on Jarvis with the advancedsettings but, without it it's the same on Jarvis. I know it's the advancedsettings that's not working.

    ALSO, The file is a H.264 ACC 720p .M3u8

    Edited once, last by dhrumil2504 (February 14, 2017 at 12:51 AM).

    • Official Post

    I've just done some tests on a RPi3 7.95.3 using Ethernet and everything is working as expected. (The RPi3 only has 736MB available due to GPU memory allocation, 250 MB it used for the system, leaving only 480MB to play with)
    However, I do wonder if Kodi has changed the algorithms for Kodi 17 as I can see no reason why the 3 x memory rule applies.

    1st test: Standard TV show
    No file - Defaults = 20MB cache readfactor 4
    Uses about 10MB ram, very little buffering - about 1 minute.
    [hr]
    136MB cache readfactor 20.
    Uses about 70MB ram - 20 minute buffered
    [hr]
    200MB cache readfactor 20
    Uses about 110MB ram - 30 minute buffered
    [hr]
    2nd test: 7G movie DTS - Network maxed out around 10,000KB/s filling the cache.
    300MB cache readfactor 20
    Uses about 180MB ram - 2/3 minutes buffered
    [hr]
    400M cache readfactor 20
    Uses about 230MB ram 5/6 minutes buffered
    [hr]
    600M cache readfactor 60
    Uses about 330MB ram 10 minutes buffered.
    [hr]
    I can't confirm that they all work reliably, but if something worked in Jarvis, try tripling the values for Krypton and see if that works.

    Just to max it: 27GB Bluray m2ts file
    600M cache readfactor 60
    Used about 450MB leaving only 30MB free - system froze (Probably out of memory)
    [hr]
    Same Bluray but over wifi. - Network maxed at 4,600KB/s
    300M cache readfactor 20
    Uses about 360MB ram <1 minute buffered. (Caused severe buffering issues (Buffer fill rate too slow)
    Over Ethernet.
    Uses about 360MB ram 3 minutes buffered (playback smooth)

    Quote from lrusak


    When you change advancedsettings.xml do you reboot each time? Or at least restart kodi to reload the new settings?

    Yes, rebooted and tried different versions of advancedsettings.xml with

    <advancedsettings>
    <network>
    <buffermode>1</buffermode>
    <cachemembuffersize>139460608</cachemembuffersize>
    <readbufferfactor>20</readbufferfactor>
    </network>
    </advancedsettings>

    or
    <cache>
    <memorysize></memorysize> <!-- number of bytes used for buffering streams in memory
    <buffermode></buffermode> <!-- Choose what to buffer:
    <readfactor></readfactor>
    </cache>

    Whatever I tried it only consumes 100Mb from 1000MB available on my pi3
    [hr]

    Thank you, will try today.

    What is your <memorysize></memorysize> value in those tests?
    Thanks

    Edited once, last by antonsn (February 14, 2017 at 1:32 PM).

    • Official Post

    @antonsn
    Use <network> for Kodi 16 and before
    Use <cache> for Kodi 17 and above.
    <memorysize> is "xxxMB cache" so convert to bytes, so 200MB is 209715200

    Also read the header - The RPi3 may have 1G ram but only about 480MB of that is available.

    [Conclusion]
    For high bit rate over Ethernet a memorysize of 314572800 and a readfactor of 20 should suffice - as long as no other processes are running and using memory, transmission/SQL etc.

    However, if multiple videos are watched, the system may still run out of memory - keep an eye on the memory usage and if it drops to less than 50MB decrease the memorysize by 50MB.

    I played around with advancedsettings.xml extensively and wasn't able to solve the buffering issues at all. I then stumbled across a thread that suggested using system mounts rather than what Kodi uses (which I think is libsmb). This has fixed the problem! No buffering at all.

    To set this up, SSH into your box and make a copy of /storage/.config/system.d/cifs.mount.sample (in the same directory) and then follow the instructions contained within it.