MyVideos121.db corrupts every 2/3 weeks

sunlollyking · May 23, 2023 at 10:01 AM

Hi All,

I posted this a few weeks ago LE 11.01 Suddenly started booting to safe mode - General Support - LibreELEC Forum whereby i ended up just resolving my performin a full clean install and manually rebuilding my database from scratch.
I've taken precautions and now backup weekly but even since then and within the time, i've had two more corruption failures of MyVideos121.db. Luckily the backup function works great and it then seems to keep LE up for another week or so but then inevitably this fails again.

Is there any diagnostics i can do to look at what could be causing this corruption, anything i could check etc?

Is it worth even trying a Kodi 21 upgrade to a nightly if there's any DB changes?

All help is appreciated,

Thanks

LybsterKodi · May 23, 2023 at 12:10 PM

Over many years I've found three main causes of database table corruption:

1. power going off whilst the database is being written to (could also be sudden loss of network connection)

2. badly formatted data being written

3. unacceptable characters

There's also hard drive sectors going duff.

No. 1 you'd probably notice

Nos. 2 & 3 are a bit more difficult but, as chewitt infers, could be being caused by a poorly written add-on but there's probably no way you can test No. 2

No. 3 you can check if you can download an SQlite browser but you need to be at least a bit of a techie. Have a look for "DB Browser for SQLite" on the web - its got an integrity check.

sunlollyking · May 24, 2023 at 10:25 AM

1) Is always a possibility.

How are 2 & 3 possible ?

I might dig into the DB viewer and have a browse around next time it corrupts.

LybsterKodi · May 24, 2023 at 12:20 PM

One brilliant way is, as I have done when wanting to test an application I've written, get a hex editor and simply alter what's stored on the disk. Great fun if you want to deliberately damage a table.

Badly written software can do it and in part it depends on how the database engine responds to say 25 characters being written to a field that is only big enough for 10, or it can write a character to a field that the engine can't handle. If running over a network a junk character can be introduced by a glitch on the network. Software writing directly to disk when it should go through the engine and overwriting some valid data. Poor transaction handling so that two lots of data are being written simultaneously and some gets in the wrong place on disk. Memory corruption which is subsequently written to disk.

Over the years I've had pretty much all of them.

What might be going on here I don't know. SQLite isn't one I'm overly familiar with (I can use SQL on it but that's because I now SQL) and I'm not familiar with the LE code base (translation I've never even looked at it).

Can one of the developers enlighten me on how the copying from MyVideos120.db to MyVideos121.db is done. If as I'm guessing its read a record, write a record then the table metadata is probably getting screwed, if its a straight copy I'm baffled.

petediscrete · May 24, 2023 at 2:40 PM

Quote from sunlollyking

1) Is always a possibility.
How are 2 & 3 possible ?
I might dig into the DB viewer and have a browse around next time it corrupts.

What medium is your database file being written to.

sunlollyking · May 25, 2023 at 8:42 AM

An M2 Drive - Holds about 1tb within an Intel NUC.

I have upgraded a few times to Kodi 21 and then back to Kodi 20 when i find that some of the addon compatibility is still a bit janky. I wonder if thre are problems in db migrations taking place.