Posts by frankvw

  • Using a RTL8812AUUSB WiFi dongle instead of built-in WiFi adaptor on RPI4

    I'm running LE11.06 on an RPI4.

    Given all the recent WiFi problems with LE10/11 on the Raspberry Pi (discussion here) I would like to try using an external WiFi dongle. Going through the junkbox I came across an Alpha Networks AWUS036AC (e.g. here). This is a dual-antenna long range model which, given that the "invalid key" problem is definitely related to lower signal strengths, might help. This dongle uses a Realtek RTL8812AU which I believe should be supported right out of the box. (Correct me if I'm wrong.)

    But how do I persuade LE to use this USB dongle rather than RPi's build-in WiFi adaptor?

    // FvW

  • ext HDs not mounted after 11.01 update

    Quote from 2469

    Hi and thanks for an interesting thread : I had a power outage yesterday, and now 2/3 disks don't want to start. Have run them in Win10 without problems after chkdsk. Today I have LE 11.0.4 generic on a PC which worked flawlessly. Any ideas? Mvh Thore from Sweden

    Until ntfsfix is included again with LE using a Windoze box to clear the 'dirty' bit on the NTFS partitions is your only option, I'm afraid.

    Quote from Reddirt

    I'm surprised at the attention this issue attracted. I for one vote to use the ext4 system for all things LE and be done with it.

    As I said above, people want to use LE because it's easy, convenient and does not require any computer skills beyond the ability to follow step-by-step instructions on how to copy a distro image to an SD card or USB stick. They are, however, stuck with USB storage devices that come out of the box with NTFS on it, and they just expect that to work.

    Which, from a UX point of view, is not all that an unreasonable standpoint.

    At this point in time any power failure or accidental unplugging of an NTFS based storage device will stop it from working, and LE has no option to fix that. You need a Linux box with ntfsfix on it just to get the partition to mount again, and a Windoze box to fix any real NTFS problems.

    Keep in mind that, to all intents and purposes, all USB storage devices come out of the box with NTFS on it.

    So as far as user experience goes, shaky NTFS support is not a good thing. Add to that the fact that this could be easily fixed to some degree (including ntfsfix again and having it triggered by a udev rule when an NTFS device is plugged in and I agree with you: we shouldn't still be just discussing this. :)

    For those with an IT background the choice between NTFS and ext4 is a no-brainer. And I have reformatted my NTFS media tank to ext4 and now all is well. But for every person who knows what ext4 is, why NTFS should be avoided in a *ix environment and how to replace NTFS with ext4, there is a gazillion "normal" people out there who just buy a device, plug it in, an expect it to work. They reason that since it does work "everywhere else" (i.e. on every Windoze box) why won't it on LE? It's not an unreasonable question, in all fairness.

    If we want to reserve LE for the IT-savvy elite then yes, let's ignore NTFS and consider its shortcomings the punisment for using it. But unless I'm very much mistaken, that's not what LE is all about, is it?

  • ext HDs not mounted after 11.01 update

    Quote from pauljw

    It still comes down to education. People shouldn't be uncleanly shutting down any device. Put LE on a ups to survive power outages and properly use the shutdown menu to power LE off. No more problem.

    You do have a point. Removing all the safeties from all equipment, removing insulation from live wires, removing the brake lights from cars, etc. will educate everyone very, very quickly and teach them not to do anything stupid with equipment that have could potential unwanted effects. But is that really the way to go?

    Let's face reality here: people are going to install LE because of its combination of simplicity and rich features, then plug any off-the-shelf USB storage device in it and expect it to work. And those things will be dismounted uncleanly sooner or later, because accidents happen. If LE then can no longer use the device their UX is impacted and they'll blame LE for not doing what they can do with their average Windoze or Linux box every single day.

    If that can be remedied fairly simply (and it can as per the above discussion) I believe that's worth doing. But that's just me.

  • ext HDs not mounted after 11.01 update

    Quote from petediscrete

    Remember the critique, lies, damn lies and statistics. Why do so many people here want to use LE I wonder 😂

    That's the point. They do want to use LE because it's easy, convenient and does not require any computer skills beyond the ability to follow step-by-step instructions on how to copy a distro image to an SD card or USB stick. They are, however, stuck with USB storage devices that come out of the box with NTFS on it, and they just expect that to work.

    As one of Debian's Elder Geeks put it (about 25 years ago now): "If the user points the gun at his foot and pulls the trigger, it is our job to ensure the bullet gets where it's supposed to." It's much the same here: people install LE for its simplicity of use, then they plug in a USB storage device that comes right out of the box, and they aren't even aware that it comes with NTFS on it and the problems associated therewith, much less are they interested in converting it to ext4. They plug and expect play.

    Now I'll grant you that there are limits to catering to all aspects of that expectation, but in this case LE's finicky response to uncleanly dismounted NTFS devices and a lack of a means to fix it within LE is serious enough for the uneducated user to consider LE broken in this regard. You and I understand what's going on, but from a UX standpoint (I'm trying to avoid using "the great unwashed masses" as a descriptive term here) this is a problem.

    Which in my opinion can be fixed fairly easily by restoring ntfsfix to the next LE distro image and (as I would prefer) hooking it into a udev rule that automatically runs it on any NTFS partition at mount time.

    Perhaps testdisk could also be added to the next image for cases in which ntfsfix is insufficient. I see that tools like udevadm (which are not exactly intended for Joe & Jane Average User) is there, so why not testdisk? It'll get used more often than the udev tools, would be my guess. :)

  • ext HDs not mounted after 11.01 update

    Quote from petediscrete

    Or else keep NTFS systems away from the Linux party. It really is a matter of re education for Windows users. With shares and other network solutions does anyone really need to be using an NTFS disc on LE anyone. And for those who do, with storage being so cheap these days a simple copy over of the contents of the NTFS disk to EXT4 would suffice before hooking up to the LE machine. Honestly what can’t be done in Linux that can be done in Windows. I prefer spending my time “consuming” my media instead of watching LE consuming my NTFS file system 😂

    I'm with you on that. In fact, when I come into power and become Ruler of All the World, the first thing I'll do is to banish Windows and NTFS from the face of the earth and make Linux skills a compulsory part of general education, and the Redmond crowd will be among the first against the wall. However, until that happy day comes around, we need to face the reality that just about every USB device sold comes with NTFS on it, and John & Jane Consumer have no idea what ext4 is, much less how to convert their newly bought USB harddisk to it. They simply plug it in and expect it to work. And most of the time it does. Except with LE, right now. And that's not a Good Thing. But it can be fixed, as per the above discussion, and it's not all that complicated. So my vote is to leave the Windows vs. Linux (and related technologies) holy War for what it is, face facts, and deal with it. :)

    Quote from HiassofT

    I'm currently looking into a different approach: let users choose to mount NTFS partitions read-only https://github.com/LibreELEC/LibreELEC.tv/pull/8308 - this would hopefully solve the issue of easy corruption in case of unclean shutdowns.

    so long,

    Hias

    If that works, I could live with it. However,it wouldn't let anyone add media through SMB or sftp, and it would break add-ons like Transmission, though. Not a big issue for me but it might be for many others. However, compromises being what they are it's better than the current unreliability of NTFS devices.

  • ext HDs not mounted after 11.01 update

    Quote from HiassofT

    Don't get too excited about ntfsfix, I did a quick test on a corrupted NTFS filesystem, removing the dirty bit, and as expected it didn't help with the errors and the ntfsdriver set the dirty bit again after mounting it because the filesystem had issues.

    You're right: ntfsfix is severly limited. It always has been. In fact, the manpage specifically states that "ntfsfix is NOT a Linux version of chkdsk. It only repairs some fundamental NTFS inconsistencies, resets the NTFS journal file and schedules an NTFS consistency check for the first boot into Windows."

    However, the issue under discussion here is that a simple unclean dismount (i.e. pulling the plug on an USB device without first dismounting it cleanly, having a power failure or other crash, or simply moving a cable with a dodgy plug or connector) will leave the "dirty bit" set on the NTFS partition regardless of whether or not anything has been corrupted. LE then refuses to remount it, ever.

    As of LE11.0.3 there's no ntfsfix at all included to reset the dirty bit. That means that as soon as you have even the slightest hiccup with and NTFS disk on LE, you need to plug the HD into either a Linux box that has ntfsfix on it, or into a Windows box and run Chkdsk. That's a bit hard to live with, and simply restoring ntfsfix to LE will go a long way to fixing that problem.

    If you have real NTFS corruption problems, no amount of ntfsfix will help you and only running Chkdsk on a Windows box will sort it out. But that's a Linux issue, not an LE issue. NTFS support for Linux has never been great (although these days it's sufficient for all regular purposes) but the lack of a proper Chkdsk alternative that can repair a corrupted NTFS has been a problem since NTFS support was first introduced in Linux.

    As petediscrete already suggested above, testdisk may be a better alternative for ntfsfix. However, for daily LE use ntfsfix will be sufficient in most cases and infinitely better than nothing at all (which LE11.0.3 currently has).

  • How to modify files in /etc?

    In the spirit of experimentation I copied /flash/SYSTEM (which is the squashfs container for the LE root fs) to my Linux box, installed the squashfs-tools package, ran unsquashfs SYSTEM which creates squashfs-root/ in which you'll find the entire contents of the root file system.

    Edit, modify, pillage and burn to your heart's content. :)

    Then use mksquashfs to recreate the SYSTEM container. This I shall cowardly leave as an exercise to the reader, because I have no idea what parameters have been used to create SYSTEM originally and what parameters should be used to recreated the modified SYSTEM squashfs container. If the LE devs can tell you, great; if not, there's trial and error in your future. If it breaks, you get to keep the pieces. :P


    Putting the SYSTEM container back is probably best done by bringing LE down, transferring the flash card from your RPI to the Linux box and copying SYSTEM to it. You could try to remount /flash rw and replace SYSTEM that way using sftp, I suppose, but I expect that would be much like trying to change the wheels on a speeding car, I suspect, and end in a similar manner.


    Personally I'm not going this route, because every LE update will overwrite the changes thus made. In general it shouldn't be necessary, either: for normal LE / Kodi use no write access to / is or should ever be required. If you do need write access, that means you're trying to do something that LE wasn't intended or designed for, and you might be better off using Raspbian instead and simply live without the LE niceties (such as using your TV remote to connect to a WiFi network, having a lot of plug-and-play options, and generally not needing a keyboard/mouse connected to your RPi ever, and all the other goodies).

  • ext HDs not mounted after 11.01 update

    Quote from chewitt

    Ahh, that explains then. Thanks for clarifying. We're contemplating where we go with changes..

    Personally I'd be happy just with the /bin/ntfsfix binary (and the /sbin/fsck.ntfs symlink to it) being restored to LE 11.0.4!

    The ntfsfix command could be linked to a button on a remote control or, since ntfsfix may not be able to solve all NTFS issues but I have yet to see it create a new one, and it's pretty quick to run, too, it might make sense to automate it. All we'd need would be a udev rule (say, /lib/udev/rules.d/99-usb-harddisk.rules) with something in it like

    Code
    ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*", ATTRS{idVendor}=="YOUR_VENDOR_ID", ATTRS{idProduct}=="YOUR_PRODUCT_ID", RUN+="/sbin/somescript.sh %k"

    The script would contain something like

    Code
    DEVICE=$1
if blkid -t TYPE="ntfs" "/dev/$DEVICE"; then
for PARTITION in $(ls /dev/$DEVICE*); do
ntfsfix "$PARTITION"
done
fi

    This isn't exactly copy-and-paste stuff of course, not to mention that it could be improved with an axe, but you see where I'm going with this.

    On second thought, a separate rule file probably isn't necessary; I've had a quick look and an addition to /lib/udev/rules.d/95-udevil-mount.rules would probably do the trick.

    [EDIT: or maybe it's 60-persistent-storage.rules - I'm not too sure at this point. I've never done a deep dig into udev rules before.]

    Anyway - happy contemplating! Looking forward to the outcome.

    In closing, thanks for all your help! It's greatly appreciated. You guys rock!

  • ext HDs not mounted after 11.01 update

    Quote from chewitt

    Copying binaries between different LE major versions or from other distros occasionally works but is never guaranteed. I'm going to have a look at Ubuntu 18.04 and what it's doing (need to find something with an Intel CPU for this though). I'm wondering if they have udev rules that run ntfs or they aliased a script that runs ntfsfix to fsck.ntfs. I'm 100% sure the util-linux tools support FAT/VFAT but do not support (and never have supported) NTFS; so the output from the fsck command needs explaining.

    As far as the fsck output on Ubuntu 18.04 is concerned: that's my bad... :blush:

    Code
    frankvw@dellfvw:~$ ls -l /sbin/fsck.ntfs
lrwxrwxrwx 1 root root 12 Sep 17  2018 /sbin/fsck.ntfs -> /bin/ntfsfix

    Sorry to have missed that.

    The version of ntfsfix on U18.04 is ntfsfix v2017.3.23. A quick grep for fsck or ntfsfix in /lib/udev/rules yields no results.

  • ext HDs not mounted after 11.01 update

    Quote from chewitt

    The tool that I added to the image is "ntfsfix" not fsck. Please test that.

    chewitt :
    Okay! That works!

    After an unclean dismount the NTFS partition cannot be made to mount again, but 'ntfsfix -d /dev/sda2' solves that problem. Progress! :)

    It doesn't directly address the fact that LE11 appears far more critical than Ubuntu 18.04 when it comes to not remounting an unclean NTFS, much less the long-standing problems of Linux vs NTFS in general, but at least it's a usable work-around!

    If ntfsfix can be included in the next point release of LE11, a udev rule to run it every time the NTFS volume is plugged in would go a long way to easing this particular pain.

    Meanwhile I've extracted the /bin/ntfsfix binary from the image and I'll add it to my LE11.0.3 install.

    Tnx!

    Update:
    Simply copying the binary across doesn't seem to work. I put it in ~/.kodi/addons/virtual.system-tools/bin because that's in the path and it's too late in the evening to start figuring out how to remount / to be writeable. But simply running it (as it did on the test image) won't work here:

    Code
    LibreELEC:~/.kodi/addons/virtual.system-tools/bin # ls -l ntfsfix
-rwxr-xr-x    1 root     root        199360 Nov  9 23:00 ntfsfix

LibreELEC:~/.kodi/addons/virtual.system-tools/bin # sh ./ntfsfix
./ntfsfix: line 3: syntax error: unterminated quoted string

    Time to call it a night.... :sleepy:

  • ext HDs not mounted after 11.01 update

    Quote from petediscrete

    Connect up the damaged disk to your Ubuntu laptop and try running Test Disk. You’ll get a far better insight into what exactly is going on.

    Thanks for the tip! That may save me having to use my wife's Windows laptop. :)

    However that's not the issue we're discussing here. LE won't mount NTFS partitions that my Linux box handles without flinching and the on-board fsck is all there is but that doesn't do much more than print its version. So one unclean dismount and you can't get things going again on LE. And that's a problem. Regardless of whether you need an external Windoze box with Chkdsk or an external Linux box with testdisk.

    Maybe testdisk could be ported to LE if fsck can't be made to do more than just print a message and exiting. But that still leaves LE's finicky behavior with NTFS.

  • ext HDs not mounted after 11.01 update

    chewitt :
    OK. I've put the test image through its paces. TL;DR version: no effect. Details below.

    First I replicated the issue on the LE11.0.3 image that I've been running for the past two weeks or so. The harddisk I'm using is a Seagate 8TB Archive SMR in external enclosure with USB3 interface (which also contains a 2-port USB3 hub). Partitions: 134MB FAT ("Microsoft reserved" rubbish that I never got around to eradicating) and the rest is NTFS.

    After a clean reboot with a previously cleanly unmounted harddisk all is well. Then I arranged for an unclean dismount of the USB harddisk. <yank> :)

    Relevant output of dmesg:

    Replugging the harddisk replicated the issue that started this whole thread: the NTFS partition is not visible from within Kodi.
    Relevant output of dmesg:

    Last two lines of mount output:

    Code
    /dev/sda2 on /var/media/Seagate_8TB type ntfs3(rw,relatime,uid=0,gid=0,fmask=37777600133,iocharset=utf8)
/dev/sdb1 on /var/media/FAT_134MB type vfat (rw,nosuid,nodev,noexec,noatime,fmask=0133,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)

    However, while mount shows the partition as mounted, it is not visible from within Kodi and /var/media/Seagate_8TB/ is empty.

    Unplugging and replugging the USB drive (which seems the first option a non-technical user would think of) doesn't fix the problem, however the mount command now doesn't show /dev/sda2 at all.

    So let's run fsck /dev/sda2:

    Code
    fsck from util-linux 2.38.1

    Which is profoundly unhelpful and doesn't fix anything. So that's the problem reproduced with the standard 11.0.3 distribution image.

    Then I did the same with the LibreELEC (chewitt): 11.80.0 (RPi4.aarch64) image as per the download link above. This shows one (and only one) difference with the LE11.0.3 distro image, which is the output of the fsck command:

    Code
    fsck from util-linux 2.39.2

    Finally I plugged the USB harddisk into my Ubuntu 18.04 laptop. (Yes, I know, I'm way overdue for upgrading. Deal with it. :)) This system runs kernel version 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux.

    Strangely enough, the NTFS partition mounted fine. I unmounted it and ran fsck /dev/sdb2 on it:

    Code
    fsck from util-linux 2.31.1
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sdb2 was processed successfully.

    I'm not sure why fsck on the old Linux box doesn't see the need to repair anything; on previous fsck runs on that partition it did come up with several messages of issues that were corrected. I wish I'd saved them at the time, but of course I didn't.

    Cleanly dismounting the USB disk from the Linux laptop did not make any difference on the LE side: when I plug it back in I still get the "volume is dirty" error in the dmesg output.

    So in summary:

    1. LE is more critical when it comes to refusing to mount a "dirty" NTFS partition, and
    2. The fsck from util-linux 2..31.1 gives more output and seems to do more than the ones in both the distro and the test LE images that I've tried.
    3. The fsck included in LE11 distro and the test image both don't seem to do much more than print its version number when applied to an NTFS partition.

    Item 1 could be kernel-related. 6.1.58 vs. 4.15.0 is not exactly trivial, although I wouldn't be able to explain why recent kernels would be less tolerant to NTFS issues... ?(

    It took plugging the HD into a Windows box and running chkdsk on it before LE would play ball with it again. Interestingly, chkdsk found no errors! But it still caused LE to mount the NTFS partition again. Uh-huh.

    I've had NTFS issues on the Linux box after a mid-write unclean dismount (files and directories that I couldn't read, write or delete anymore from Linux) and that needed a Windows box to fix. But the partition would still mount. LE seems much more picky.

    And that's it from me for today. Stay tuned - film at eleven... :P

  • ext HDs not mounted after 11.01 update

    I've downloaded it, but I'm not sure what to do. I'm not a hard-core developer and I'm not well-versed when it comes to things like Github commits. I work on in-house PHP website back-ends for a living. The above tar doesn't seem to be an image but contains a file system, so I'm not too sure how to install it. To have a look at the stable release image, thinking that the contents of the tar might replace some of it, I tried to download the current generic image for generic PC (no Nvidia support) which should in theory work on my Dell Inspiron 15 3576 laptop but that results in an error message:

    Code
    redigo: nil returned

    So I'm not sure how to proceed.

    I run LE 20.0.3 on an RPi4 (on which I experienced the above NTFS issues) and I've got my Linux laptop for work (still running Ubuntu 18.04, and I can't install LE on that one because I can't risk nuking my work system). That's pretty much it. Virtualization isn't really an option for testing NTFS issues since the virtualization is more than likely to skew the results.

    So. How now, brown cow? :)

  • ext HDs not mounted after 11.01 update

    Quote from chewitt

    ^ LE uses the full e2fsck binary not busybox.

    I built a Generic/GBM image here: https://chewitt.libreelec.tv/testing/LibreE…_64-11.80.0.tar which should reinstate mkfs.ntfs and include the ntfsfix tool from ntfsprogs. The image is not tested since I don't have any Intel CPU hardware around these days, but please have a play with the binaries to ensure they compiled okay and to see if ntfsfix can resolve the dirty-filesystem issues you're seeing?

    I'm on the road this week; I can only download the image on Friday afternoon. Will revert then and let you know what happens. Meanwhile thank you so much for your support; you, sir, rock! :)

  • ext HDs not mounted after 11.01 update

    Quote from chewitt

    Yup. And the utility that's been dropped with NTFS-3G is "mkfs.ntfs" not "fsck.ntfs" .. and if you do some reading around this topic there is a conspicuous absence of fsck or similar tools for NTFS on Linux. There's an older "ntfsfix" which simply clears the dirty flag; which will probably allow the drive to mount in some/most cases, but risks causing damage in scenarios when proper chkdsk.exe is required to resolve a more serious problem with the filesystem (and due to that, isn't something we'd try to use). It looks like we (and the the rest of the Linux using world) is needing Paragon to clean-up and submit their fsck tool code.

    Agreed! But I'm not holding my breath.

    On my current (old Ubuntu 18.04) Linux laptop there is a /sbin/fsck.ntfs but that's just a symlink to /bin/ntfsfix, the manpage of which is quite honest about what it doesn't promise: "ntfsfix is a utility that fixes some common NTFS problems; ntfsfix is NOT a Linux version of chkdsk. It only repairs some fundamental NTFS inconsistencies, resets the NTFS journal file and schedules an NTFS consistency check for the first boot into Windows." Relevant command line options are limited to -b (Clear the list of bad sectors) and -d (Clear the volume dirty flag if the volume can be fixed and mounted). And that's it - not a whole heck of a lot. But as of right now that's about as good as it gets.

    And the fsck implementation in busybox (which I believe to be the one in LE, although I might be wrong) looks to be even more limited.


    In short, it looks like NTFS on LE11 is even less of an option than it is on other Linux platforms, and although I would say that from a UX standpoint proper and robust NTFS support is non-optional, I'm not sure it's realistically feasible at this point in time. :(