ERROR: cannot read `./usr/lib/libsqlite3.so.0.8.6'

    My libreelec has suddenly broken down. Kodi won't start up. It seems the system is mostly working correctly except for Kodi. Looking at its journal it gives:

    Code
    Nov 30 14:36:30 LibreELEC systemd[1]: Stopped Kodi Media Center.
Nov 30 14:36:30 LibreELEC systemd[1]: Starting Kodi Media Center...
Nov 30 14:36:30 LibreELEC systemd[1]: Started Kodi Media Center.
Nov 30 14:36:30 LibreELEC kodi.sh[8232]: /usr/lib/kodi/kodi.bin: error while loading shared libraries: /usr/lib/libsqlite3.so.0: cannot read file data: Input/output error

    I had a look at all the files and it seems that file is the only one causing I/O errors:

    Code
    # head -n1 /usr/lib/libsqlite3.so.0.8.6
head: /usr/lib/libsqlite3.so.0.8.6: Input/output error

    Any ideas how I might be able to fix this? I tried updating libreelec, but that didn't work.

    I'd guess the boot media (SD card?) has bad sectors and that file can no longer be read, which breaks Kodi. Or the filesystem got messed up, but that will normally be fixed automagically on boot, so an underlying storage issue is far more likely. SD/USB media doesn't last forever and even HDD/SSD die eventually. My $0.02 is to backup essential data while you still can.

    Thanks for your response. The SSD is relatively new. Like less than 6 months. In addition, that's the only file that has failed, so was kind of hoping for a way to simply replace that file, and it would work. I dunno. I've already backed up /storage. If there is no way of salvaging, I'll try to re-install it and if it happens again I'll have to replace the SSD.

    Is it all possible to force an update to replace that file? Or perhaps I can mount the disk rw, replace it, and then try?

    Quote from praaattee

    Is it all possible to force an update to replace that file?

    Yes, you would drop an update file (.tar or .img) in /storage/.update and reboot. But that won't fix your underlying issue, and likely would fail the update. You really need to investigate why you have bad blocks, i.e. some sort of surface scan/badblocks and resolve the uncorrectable errors. I don't know much about SSDs, never owned one, but if it has SMART then maybe initiate a extended or long test via SMART and review the device information when it completes -- no idea if that will work for an SSD. Maybe someone else with failing SSD experience will chime in to clarify the best approach for you.

    Quote from praaattee

    Or perhaps I can mount the disk rw, replace it, and then try?

    Not really, it's a squashfs filesystem mounted via loopback, so it's essentially a compressed single file on the /flash partition that is uncompressed and presented as a read-only filesystem. The /flash/SYSTEM file is that squashfs filesystem. As mentioned above, none of this would resolve your underlying problem and likely would make things worse, got to fix the root cause (which may mean a new SSD).

    So I did a scan using badblocks, and did a SMART scan, which returned nothing:

    Code
    $ sudo badblocks -v /dev/sda
Checking blocks 0 to 234438655
Checking for bad blocks (read-only test):
done
Pass completed, 0 bad blocks found. (0/0/0 errors)
    Code
    $ sudo smartctl -H /dev/sda
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.19.0-76051900-generic] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-he alth self-assessment test result: PASSED

    Side note: apparently "he alth" is a censored word, so had to add a space to it in the last line of smartctl.

    Post a full log - it could also be some compatibility issue with your USB SSD adapter, power supply issues or anything else

    Please provide a full debug log.

    How to post a log (wiki)

    1. Enable debugging in Settings>System Settings>Logging
    2. Restart Kodi
    3. Replicate the problem
    4. Generate a log URL (do not post/upload logs to the forum)

    use "Settings > LibreELEC > System > Paste system logs" or run "pastekodi" over SSH, then post the URL link
    Quote from praaattee

    $ sudo smartctl -H /dev/sda

    This just reports the overall he alth status, you would want to initiate a long or extended test via smartctl -t long, and then review the smart information with smartctl -l selftest, and probably review smartctl -a to see if there are any recorded errors (bad blocks).

    Having started on the path of "bad storage" I'not sure it's a correct diagnosis. LE packages the userspace side of the OS into a single squashfs file (SYSTEM) which is decompressed into a virtual (read-only) filesystem in memory. This means that to successfully decompress the SYSTEM file we have to read all the data for SYSTEM from boot media. If there are bad blocks the decompression of SYSTEM and boot fails; LE is not the same as a conventional distro where a bad block impacts only the specific file(s) written on the bad block.

    NB: As we decompress SYSTEM into RAM on systems with >=1GB RAM, you can get similar effects to bad boot media with bad RAM.

    It's not clear what kind of hardware you're using, but I would deliberately unplug the SSD and boot from an LE image on SD/USB media to rule out RAM or boot media issues. Also, boot logs (dmesg) would be informative.

    Quote from HiassofT

    Post a full log - it could also be some compatibility issue with your USB SSD adapter, power supply issues or anything else

    Please provide a full debug log.

    How to post a log (wiki)

    1. Enable debugging in Settings>System Settings>Logging
    2. Restart Kodi
    3. Replicate the problem
    4. Generate a log URL (do not post/upload logs to the forum)

    use "Settings > LibreELEC > System > Paste system logs" or run "pastekodi" over SSH, then post the URL link

    Unfortunately, I am unable to get full debug logs because Kodi does not start.

    Quote from chewitt

    It's not clear what kind of hardware you're using, but I would deliberately unplug the SSD and boot from an LE image on SD/USB media to rule out RAM or boot media issues. Also, boot logs (dmesg) would be informative.

    So I'm running this on a Raspberry Pi 4 4GB.

    And well, while trying to get the dmesg this time, I got a message on the screen saying "filesystem corruption detected" and I should shut down for an automated fix. Not sure why this wouldn't have come up the countless other times that I've restarted it. Anyway, I shut it down, started it back up, and it's working again. Didn't change anything.

    Is there anything I should consider now that I'm back up and running. I've already made a backup of /storage. If anyone has any thoughts on preventative things to do here, that would be great.