Entire Folder Emptied!

    Hoping someone can help here...

    I have been setting up a new RPi 4 and latest LE. Coming from an older RPi 3 running a version of LE several major revisions old (like 8 or 9 or something). I migrated my library, set up sources, connected my USB drives to the new Pi...all seemed fine.

    I connected to one of the attached NTFS formatted hard drives over SMB from my Mac (as I have done hundreds of times with my RPi 3) to move a few movie files. After the move (to a master 'Movies' folder) I viewed that folder and noticed it was showing only about 80 files...there should be like 600+!

    Then I went to the TV and tried to play some movies...and they all say the source cannot be found. Looking in File Manager, it showed the folder was empty.

    I safely removed the drive in Kodi, connected it to my Mac, and that folder shows empty on my Mac too. Not a single file shows. All other folders still have their contents, but not that one.

    Get Info on the folder says it's zero k, should be over 2.5TB. That space has not been freed up on the drive though, Info on that says only a few hundred GB free.

    It seems like something happened when I connected over SMB and it's nuked my movie collection! I never unplug the drives without using 'remove safely' or shutting down from Kodi first.

    Are there any known issues that might have caused this? Really freaking out, hoping someone can help =O

    Cheers

    Edited once, last by symean (August 24, 2023 at 8:44 AM).

    Ok so after my anxiety attack I ram chkdsk on a Windows laptop that revealed ~800 files ‘orphaned’, and it managed to repair it somehow within a couple minutes. Now shows correctly in the file manager in Kodi and movies play again.

    Still though, I am worried this seems very coincidental that it happened on a new Pi and new version of LE and Kodi…really keen to hear if anyone else has experienced this.

    Cheers :)

    Old LE version will be using the NTFS-3G userspace filesystem driver and current LE version will be using a new in-kernel NTFS filesystem driver. That will explain differences in how things are mounted and perhaps different error messages. It doesn't explain how things got corrupted in the first place; but with USB media PSU's and or the simple fact of NTFS + removable media = /shrug

    Quote from chewitt

    ...the simple fact of NTFS + removable media = /shrug

    Thanks, Would ExFAT be a smarter file system choice then? While we’re at it, for network file management is there a smarter choice than SMB?

    NTFS/SMB was working fine, but obviously something ‘new’ has upset the delicate balance :) whatever is the most reliable/least prone to errors...

    I'd use EXT4 for Linux filesystems and simply use the network to transfer files; thus avoiding the dependency on exFAT or NTFS for Windows compatibility. There's nothing wrong with Samba/SMB for the server. We don't enable/provide NFS. You can always use an SFTP (SCP) client to transfer files over SSH.

    It's Mac compatibility I was considering. I prefer to use my network to transfer files as I do it sometimes several times a week and I don't want to plug/unplug the USB that often, however I'm setting up one for my parents who will direct connect it to their Mac every other month.

    You can add full EXT4 support to MacOS with third party software, I may test that today.

    After a lot of Googling I am reading about issues with SMB on the latest MacOS Ventura. No one has quite nailed it down to a specific cause, though much of the talk seems to indicate SMB3 is not rock solid. Some people have had luck forcing SMB2-only mode as a workaround. But then I've had no issues connecting over SMB to the NTFS formatted drive via a RPi3 for years...

    I know I'm jumping several major versions in LE and moving to a RPi4, so it's not easy to nail down the culprit. There might not even be an issue to worry about, perhaps it was a one-off.

    I am going to do some write/read/move/delete tests on a portable drive today over SMB connected to both the RPi3 and RPi4 in both ExFAT and NTFS, and perhaps EXT4. Might even force SMB2 on the Pi if I can reproduce the problem...

    Thanks for the help :)

    On your end you don't really need the compatibility so use EXT4 on the drive and perhaps Transmit to shift files. SFTP is slower than NFS/SMB but the client will queue files and transfer things reliably. On the parental end .. exFAT is really the only option for plug/play with macOS unless you add third-party drivers to the OS.

    The best investment(s) I made for reliable playback were some Synology NAS devices in the network. Simple to copy media to after ripping etc. and the LE device becomes a simple/dumb playback client accessing content from an SMB share and with nothing dangling/attached to the RPi(4). On the parental end I have a similar arrangement where the NAS also acts as the media device, but also a local TimeMachine destination so I don't rely upon elderly minds remembering to make backup copies on a USB device (they are good at hoarding data but backup habits evade them). Once in a while we copy new data from removable drives to the parental NAS where there's disk redundancy and also remote sync of specific dirs back to the NAS at mine to effect some off-site backup of important files.

    I think I'll go for ExFAT for my parents, they don't get new files often so direct connecting it once every few months is not a big deal.

    For me, I'm convinced I need to move to a NAS, and the Synology ones have always looked pretty good. That will have to be stage 2 though, I'll have to think about drive brand/capacity, raid type, learning Synology's OS...don't want to pile all that onto what's going on!

    I never had a copy of the media because it took up so much space, but I found a drive from a server I'm going to use to copy it all. Then I'll just get the existing NTFS drives stable on the new media player, and then I can relax and start looking into the NAS.

    Thanks for the advice, really appreciate it :)