Posts by milhouse

    will be apply correctly?

    Yes - so long as your movies and tv shows have the same names before and after (which they should). The script doesn't actually care where they are (or were) located when restoring the watched status, it only matches on the movie or tvshow names.

    then let me know if the order of steps is correct:

    Yep, you got it.

    [of course I need to know how to install the script on my existing libreELEC.

    It's explained in the first post of the thread I linked.


    If you want the super simple version, log in to LibreELEC and run:

    Code
    1. curl https://raw.githubusercontent.com/MilhouseVH/texturecache.py/master/texturecache.py -o /storage/texturecache.py
    2. chmod +x /storage/texturecache.py

    Then to backup your watched status:

    Code
    1. /storage/texturecache.py watched movies backup /storage/watched-movies.dat
    2. /storage/texturecache.py watched tvshows backup /storage/watched-tvshows.dat

    and to restore watched status (after you have rescraped the new library):

    Code
    1. /storage/texturecache.py watched movies restore /storage/watched-movies.dat
    2. /storage/texturecache.py watched tvshows restore /storage/watched-tvshows.dat


    then what do I do on my other kodi instances besides pointing them to the new external database?

    Point them to the MySQL database by copying advancedsettings.xml.


    I'd also suggest copying sources.xml (in the same directory as advancedsettings.xml) from the LibreELEC client to your other clients just so that you have the same sources on all your clients. This step probably isn't essential, particularly if you don't plan on doing any scraping on those clients, but it may be useful.

    Do I need to do something else to get the watched count etc on those boxes as well? Or will that start coming from the database?

    Nope, that will all come from the central MySQL database.


    Once you have scraped your library on the LibreELEC machine you may find that the other non-LibreELEC clients don't have any artwork cached. They will cache the artwork as it is displayed, but this can mean navigating the GUI on those clients will be quite slow. A solution is to use texturecache.py running on the LibreELEC client to automate the caching of artwork on your other clients, but that's probably a story for another time (or read the first post of the linked thread, the details are all there, pretty much).

    Milhouse that is too advanced for me.

    I'm really pretty sure it's not, particularly if you're willing to persist with path subs which are far more complicated IMHO! :)


    To use nfs:// paths just create an nfs:// source, in your case you'd create a Movies source with nfs://192.168.1.121/volume1/video/ and a TV Shows source with nfs://192.168.1.121/volume1/TV, then scrape the contents of each source into your new library.


    I'm sure you'll get it working with path subs eventually, but you'll always have a schizophrenic database that will try and break itself at every opportunity. Best of luck! :)

    So you have these two path subs:

    Code
    1. 21:52:34.954 T:18446744072517099920 DEBUG: Configuring path substitutions
    2. 21:52:34.954 T:18446744072517099920 DEBUG: Registering substition pair:
    3. 21:52:34.955 T:18446744072517099920 DEBUG: From: [/storage/videos/]
    4. 21:52:34.955 T:18446744072517099920 DEBUG: To: [nfs://192.168.1.121/volume1/video/]
    5. 21:52:34.955 T:18446744072517099920 DEBUG: Registering substition pair:
    6. 21:52:34.955 T:18446744072517099920 DEBUG: From: [/storage/tvshows/]
    7. 21:52:34.955 T:18446744072517099920 DEBUG: To: [nfs://192.168.1.121/volume1/TV/]

    but they're not working for your artwork as Kodi continues to retrieve the artwork files using the "local" path:

    Code
    1. 21:52:39.020 T:18446744072529132944 DEBUG: GetImageHash - unable to stat url /storage/downloads/xbmc_videodb_2018-07-13/movies/Toy_Story_3_2010-poster.jpg

    What you'll need to do is add yet another path sub for /storage/downloads.


    I'm just trying to get it working as I don't want to loose my watched status

    If you just want to save watched status (ie. watched state, resume point and play count) then might I suggest using texturecache.py: [RELEASE] Texture Cache Maintenance utility


    You can install the script very easily into LibreELEC. In fact I include it as standard in my own LE9 test builds.


    To backup your watched status, on the LibreELEC client:

    Code
    1. texturecache.py watched movies backup /storage/watched-movies.dat
    2. texturecache.py watched tvshows backup /storage/watched-tvshows.dat

    Now create your new library in MySQL using only nfs:// paths, then restore the watched status to the MySQL database:

    Code
    1. texturecache.py watched movies restore /storage/watched-movies.dat
    2. texturecache.py watched tvshows restore /storage/watched-tvshows.dat

    Are you using Yatse as well, because some of the JSON requests in your log look like the polling spam typical of Yatse. If you are using Yatse, please disable it while testing with your Tuefel, as they could be interacting with each depending on how badly the apps have been written. If you are not using Yatse then maybe the Tuefel app has the same design inefficiency as Yatse.


    In your kodi.old.log I can see (once I filter out the spam) that an app - presumably the Tuefel, maybe Yatse - is requesting song details in batches of 975 songs. It manages to request 16750 songs, then stops at 09:18:04.581- there are no further JSON requests (other than spam) which suggests the Tuefel is still working. Kodi doesn't appear to have crashed at this point as it is still processing the spam requests.


    Someone (Teufel? Yatse?) starts a music library scan at 09:20:21.511, which runs for several hours, finishing at 14:37:30.579.


    Then, at 16:30:24.536, there is a UPnP request (presumably from the Tuefel?) which seems to be requesting all songs - 49621 in total, in batches of 100. Kodi seems to responsd OK with the first 400 items, but then hangs when being asked to process the 5th batch. My guess is there is a serious memory leak (or scale issue) in Kodi's UPnP implementation, which is a highly experimental feature in Kodi 18.


    Please open a trac ticket on Kodi - TRAC and include the link to kodi.old.log - this will need the Kodi UPnP developer to investigate further.

    Post your kodi debug log from the Firestick when using path substitution, I can't really comment further without knowing what the actual error is. Do the movies play successfully on the Firestick when using path substitution?


    Note that if you perform a library scan on the Firestick which added new content to the database, chances are the nfs:// paths will be added to the database so you'll end up with a mixture of /storage and nfs:// paths in your database. The nfs:// paths will probably still be accessible by the LibreELEC device (even without path substitution) but from an OCD perspective this would drive me up the wall... you could probably have re-scraped a clean, 100% nfs:// database by now.


    I'm not sure why you don't want to start afresh, but if you don't want to rescrape because you want to retain your watched statuses then there's a very simple solution for that which does not involve exporting the database. Also, exporting artwork is (IMHO) a really bad idea as the quality of the exported artwork is so much lower than the original artwork - again there is a solution for converting remote artwork to local artwork without losing quality. Unfortunately both solutions involve a command line, but if you're comfortable with that then you shouldn't have too many problems. I can post more details if this is of interest to you.

    Yes, the simple way is to drop the database and start again using network shares! :)


    Everything else, including path substitution, is nothing but a barely working workaround, as you're discovering.


    I don't know how you've set up your path substitution but the problems with artwork are most likely because the artwork locations are once again inaccessible to the remote clients. You might be able to add more path substitution rules for the artwork, maybe. I personally wouldn't bother with path sub - you'll be dealing with this issue forever in one form or another, bite the bullet and fix it properly...

    openssl does use this file by default, but it will be trying to access /etc/ssl/cacert.pem - we seem to have dropped the ca filename prefix (creating cert.pem not cacert.pem) in the transition back from libressl.


    openssl example using current LE master (bogus private key, just for testing):

    Code
    1. rpi22:~ # openssl ca -keyfile /storage/.config/private/cakey.pem
    2. Using configuration from /etc/ssl/openssl.cnf
    3. Error opening CA certificate /etc/ssl/cacert.pem
    4. 1996051232:error:02001002:system library:fopen:No such file or directory:bss_file.c:406:fopen('/etc/ssl/cacert.pem','r')
    5. 1996051232:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:408:
    6. unable to load certificate

    But you need to leave /etc/ssl/cert.pem because this is default name used. Use symbolic link instead.

    I'm not so sure - the openssl config is referencing /etc/ssl/cacert.pem not /etc/ssl/cert.pem, so it looks like /etc/ssl/cert.pem is a misconfiguration on our part. It has probably never been an issue as curl is hardcoded to use the /etc/ssl/cert.pem path.

    Also I would rename openssl-config to something more appropriate.

    It's not the greatest name I grant you, but it's consistent with our other kodi-config, smbd-config etc. scripts that all do something similar.

    vpeter what do you think of this change? Comparing LibreELEC:master...milhousevh:le90_user_ca · LibreELEC/LibreELEC.tv · GitHub


    It should allow a user to safely add *additional* CAs in /storage/.config/cacert.pem without losing the benefit of existing system-supplied CA certs.


    Replacing the shipped system CA cert file with a user CA file is going to result in long-term issues as CA certs expire or are compromised/replaced, and users with custom CA files start reporting weird and unreproducible certificate-related issues with public websites because their CA file is out of date.


    I've also reverted the name of /etc/ssl/cert.pem to /etc/ssl/cacert.pem as per the contents of /etc/ssl/openssl.cnf.

    It seems the Android addon is grinding it's way through your database using UPnP.


    What makes you think it runs out of memory - you've said "Memory is getting more and more low up to 40140 kb", how are you determining this?


    Unfortunately I can't see the journalctl log you posted to ix.io (the service is down at this time) but if Kodi is not restarting then it's probably not being killed by the out of memory process killer.


    From the kodi.old.log it looks like 1,111 UPnP items are processed in approximately 3 minutes. Then the "bing" screensaver starts. I don't see any further processing of any kind after that.


    It might be worth testing without any additional addons installed - start with a "clean" .kodi folder:


    1. systemctl stop kodi

    2. mv /storage/.kodi /storage/.kodi.bak

    3. reboot
    4. Add your music

    5. Run the Android addon


    Once you are done testing:

    1. systemctl stop kodi

    2. rm /storage/.kodi

    3. mv /storage/.kodi.bak /storage/.kodi

    4. reboot

    With #0708, does Kodi actually crash due to out of memory (it may take a while, up to 30 minutes or so)? Once Kodi restarts, post kodi.old.log and also the contents of "journalctl -a".


    It looks like the Teufel app is hammering on the Kodi webserver for information. Maybe this is causing a memory leak, or the Teufel is just flooding Kodi with requests.


    You could try enabling debug logging, and then also enabling JSON-RPC and Webserver component logging, and then start your Teufel and maybe we'll see what requests the Teufel is making to Kodi. Assuming Kodi restarts, upload kodi.old.log and "journalctl -a".

    Try looking at an existing addon.


    If your add-on is a binary add-on - ie. it contains C/C++ code - then look (for example) at pvr.demo, and if not - ie. it's 100% Python and/or XML - then look at something like the LibreELEC-settings package.


    You need to add the package (ie. package.mk) to the build, with the package.mk linking to your add-on source repo, then you need the package to be included in the build by adding the package name as a "PKG_DEPENDS_TARGET" in packages/virtual/mediacenter/package.mk