Mem leak on 7.95.3, 8.0 & Milhouse #0303 RPi3

1st Official Post

    Hi there, have a repeating mem leak while adding music source. Am about the half source in (60MB DB size)

    Here's a journal:
    OVEU

    The debug log shows no issues, so I'll keep it for the moment.

    No crashlog either, as kodi gets restarted (see journal) or simply hangs

    Edited once, last by uatschitchun (March 6, 2017 at 9:27 AM).

    Are you using SQLite or MySQL (you see, your log would have given us all the technical details we now need to ask you).

    Does it always get to the same point in the scan before running out of memory, or are new songs/albums added with each scan before again running out of memory?

    How big is your music media library (number of songs)? Do you have embedded artwork or are the posters/fanart separate, and if so how big are these images (width/height)?

    If you're already in contact with Dave (Blake?) then he's probably the best person to investigate this assuming it's a core Kodi issue - presumably you've hit upon a very memory inefficient code path, or some aspect of the scan - maybe tag processing - has gone bonkers.

    And crashlogs won't be created when the Kodi process is killed by the out of memory (OOM) process killer.

    Ok, so after I got my music source scanned in (did switch from Mysql to a recent MariaDB which speeds up music scanning from 19h to 5h) the mem leak is definitely connected to using Xonfluence (guess it's skin.widget addon or library data provider, which come along with Xonfluence) together with a huge music source (130k files).

    I'll provide logs!

    Same setup on my s905x box is working great... so it's only the RPi3...

    On the mem leak, this is the last entry in mysql.log:


    And this is kodi.log
    jGTU

    Here's a video
    Dropbox - VID_20170227_221841.mp4

    Question:
    How do I set MALLOC_MMAP_THRESHOLD by hand on RPi? It gets set from within kodi-config for 64bit, but DaveBlake wanted me to test with MALLOC_MMAP_THRESHOLD_=32768
    kodi-config is read only and /run/libreelec/kodi.conf gets overwritten each time kodi starts, so I couldn't set it there.
    I tried setting it as environment variable and

    Code
    echo $MALLOC_MMAP_THRESHOLD_
32768


    make me believe I've done it right.
    Any other way?

    This is Milhouse #303 running since yesterday evening.
    LDP & skin.widget together with Xonfluence (tricked it into with adapting xbmc.gui version from required 5.12 to 5.13).
    This was running over night and produced 9 crashlogs and at least 1 reboot :s

    Edited once, last by uatschitchun (March 6, 2017 at 7:08 AM).

    Pls, couldn't anyone tell what's needed to get this one pinned down?

    Dave and me are down to it being caused by Library Data Provider and/or Skin.Widget (both coming along with Xonfluence) and the regular queries done by that addons to the DB. MyMusic DB (MariaDB) is quite large 120k songs...
    Have had problems with mysql, so I switched to MariaDB as it's alot faster!

    Latest Milhouse (#0305) is better in the matter of "leaking" (not that fast as in the video), but there are still crashes (see above)

    I'm checking against Estuary with LDP & S.W running to see if it has an influence.

    No problem running Xonfluence with LDP & S.W on Linux (14.04) or Amlogic (kszaq) against my DB, only RPi3!

    Edited once, last by uatschitchun (March 6, 2017 at 8:32 PM).

    No, sadly no real solution. As from the tests we did, best is to not have any addon querying the db (like Library Data Provider).

    Dave did some changes which are added in kodi 17.1.

    Have you've got a little more info on your setup?

    Edited once, last by uatschitchun (March 31, 2017 at 12:55 PM).

    My system has a crash every 90 minutes. Attached is a log:

    Do you need more information?