RPi4 & HDD corruption

    I've had corruption on my 6TB HDD again. I had been looking at different skins and I had rebooted / powered off the Pi a few times and I had made changes to the data on the HDD so my latest wondering is does LibreElec flush the buffers to a USB connected drive and wait for that to complete before either rebooting or powering off?

    So far I've managed at best a week without corruption so for that and an issue with playing back recordings from TVHeadend I've sent the Pi on sabbatical for a few months.

    Thanks to those who've helped me so far and as Arnie says "I'll be back" ^^

    I was talking about A/V streaming buffer, because you didn't mentioned a specific buffer. Kodi's default setting includes Samba caching into RAM. Maybe you have to increase the "memorysize" parameter to keep the whole Samba cache in RAM.

    Quote from Da Flex

    I was talking about A/V streaming buffer, because you didn't mentioned a specific buffer. Kodi's default setting includes Samba caching into RAM. Maybe you have to increase the "memorysize" parameter to keep the whole Samba cache in RAM.

    I was just asking about any buffer that might need flushing to an external USB drive. Something causes LibreElec/Kodi/RPi4 to corrupt an attached 6TB HDD. This does not happen with the RPi3 and its older s/w. What I need is to identify why so that I have a chance of stopping it happening and I can switch over to the RPi4.

    So back to the question - does LibreElec/Kodi/RPi4 flush any outstanding buffers to an external USB HDD before rebooting or powering off?

    Thanks for trying to help. I realise I should have posted the configuration as I did in the earlier posts about this. Its a 4GB RPI4 with a 120GB SSD or LibreElec & Kodi with a self powered 6TB HDD for films, both plugged into USB 3 ports, dongle for USB mini keyboard plugged into a USB 3 port on the 6TB HDD& Haupage dual tuner plugged into USB 2 port with TVHeadend. Connected via powerline ethernet to a mixed Windows (3 laptops, 1 desktop all on powerline) & Linux (1 laptop using wifi) LAN. I transfer stuff to the 6TB HDD via SAMBA.

    The RPi3 setup had two RPi3s both running off microSD cards, one with the Haupage tuner and the other HDMI to the TV, both plugged into powerline.

    With the RPi3 setup, started back in 2018, I have not experienced corruption, with the RPi4 setup I don't think it even managed a week without corruption.

    Basically I'm still trying to understand why so I can stop it happening and start using the RPi4.

    Have you tried (for testing purposes ) to run the RPi4 from sdcard and only the 6TB HDD attached ?

    That's what i would do first because it would take out some variables from the equation (especially regarding multiple USB devices connected to the RPi4 ).

    BTW ,your 6TB is still on NTFS?

    Yup - still NTFS.

    I have thought of swapping to an SD card since that is one of the significant differences between the two setups, and one I'm willing to alter - no way am I replacing the nice stable s/w on the Pi3s. I'm recording the Rugby World Cup happening in NZ for a friend so I have time to buy an SD card from good old Amazon.

    I'll report back after the rugby.

    Went into a charity shop this afternoon and bought a 2nd hand TV so I can now set up the Pi4 and mess around. Just updating my backup 6TB HDD so I can add that on. About the only thing different will be the fact I don't have a spare Haupage tuner. I do have a single tuner so I'll add that in to copy the setup I have problems with.

    As is usual with computers just sit back and watch everything work perfectly!

    Quote from Da Flex

    You could add a timeout to your poweroff / reboot. Maybe that's needed to complete buffer writings.

    After some experimentation I think this is (sort of) the problem. I renamed a file, left it about 10 minutes then pulled the plug - corruption occurred. I think its the proper reboot/power off that's needed.

    Can anyone on the team let me know how frequently buffers would be flushed to disk or if its only on reboot / power off.

    Quote from ghtester

    Didn't you try the sync command before unplugging the USB storage?

    See for instance this webpage: https://www.kaper.com/notes/linux-sy…or-copy-to-usb/

    Not yet. I've just been experimenting to see what was causing the corruption. If running sync works it will be good but a bit inconvenient since I'll have to ssh into the Raspberry Pi to run it. I need to run a few other experiments and have a look through the addons yet.

    The results so far are that sync doesn't work. I tried altering a couple of file names, ssh into the RPi and ran sync, ran the watch to show status - no dirty bits, turned off, turned on - Kodi will not access the disk.

    The good news is that reboot or power off do seem to flush things to disk.

    I'm not sure whether this is something for LibreElec or Kodi. If Kodi please let me know and I'll post over there.

    I've now tried SAMBA and FileZilla for transferring files to the RPi4 and SAMBA to rename files on the RPi4. I've tried turning my PC off after the transfer to make sure the share is disconnected. I've tried logging in to the RPi4 via SSH and running sync. I've tried waiting a while, about 10-15 minutes and overnight, just to see if the disk would get updated and the answer in all cases is no - it doesn't.

    Rebooting, or powering off through the menu does flush things down to disk but its a bit of a pain to have to do this (and remember to have to do this) after any alteration, or addition to the disk.

    I can appreciate, as Da Flex, posted that not writing down was to help prolong the life of MicroSD cards but could a switch be added to allow it to happen?

    The reason I'm wondering if this is a Kodi issue is that if I change skin this is also not stored (or from my simple test) unless the Pi is rebooted.