RPi NOOBS partition information

1st Official Post

    I'm the developer of PINN, a fork of NOOBS for the Raspberry Pi.

    I've noticed that the partition label names for LibreELEC in LibreELEC.tv/partitions.json at master · LibreELEC/LibreELEC.tv · GitHub are too long to be used as partition/volume labels. They currently have names like "LibreELEC_RPi2_System" and "LibreELEC_RPi2_Storage" derived from your distroname and project name.

    However, FAT partitions have a maximum label name of 11 characters, and 16 for ext partitions.

    NOOBS may also add a single digit to the label name to make it unique if necessary, further restricting the original name length to 10/15 characters.

    In addition, NOOBS will ignore any partition label >15 characters. This is what is happening in the case of LibreELEC and its partitions are not being labelled at all.

    This seems a bit of a bug in NOOBS, but the developers have indicated they are not planning any changes to NOOBS in the near future.

    So I suggest the partition labels used for LibreELEC are shortened to 10 characters so that their partitions are labelled correctly.

    I shall try to put a fix in PINN to workaround such issues, but even so it is not clear how to do this universally. For example, If the existing label names are truncated to 10 characters, both LibreELEC partitions will be given the same name before the single digit suffix is added to disambiguate them. Maybe LE2System and LE2Storage would be appropriate? But I'll leave it to you as it seems to revolve around some macro system. Whatever you choose, please ensure consistency with their associated tar files.

    In NOOBS, this is largely a cosmetic issue, but it makes identifying the partitions much easier if they have the correct names.

    In PINN it is blocking a new feature, so I will have to work around it somehow, but I'm sure it would be better if the partitions could be labelled consistently in the first place.

    Thanks.

    Does the filename used for the tarfiles, here:

    LibreELEC.tv/image at master · LibreELEC/LibreELEC.tv · GitHub

    have to match the partition label names?

    I'd be inclined to just use "LEStorage" and "LESystem" as the label names, and leave everything else the same. But I don't think any of us on the team are NOOBS users so we'll have to take guidance on this.

    The NOOBS usage of ${PROJECT} is all wrong anyway as it doesn't take ${DEVICE} into account.

    Hi Milhouse, Thanks for getting back to me (and sorry for the late reply as I forgot to turn on email notifications for this forum :blush: )

    Yes, the tar files need to match the partition label names, as this is how NOOBS & PINN identify the tar file to be installed for each partition.

    "LEStorage" and "LESystem" would be fine.

    When performing a modification for NOOBS, please bear the following in mind:

    NOOBS can install LibreELEC from the internet, or from a local device (if the NOOBS distribution files have been previously downloaded).

    In the case of a local installation, it uses the default filename derived from the label specified in the "partitions.json" file.

    For an internet installation, it uses the tarball URL given in "http://downloads.raspberrypi.org/os_list_v3.json" which currently specifies the existing tarfile URLs such as: "http://releases.libreelec.tv/noobs/LibreELE…2_System.tar.xz" et al.

    So, if you change the partition names and associated tar file names, you need to notify the Raspberry Pi developers of this change so they can update their tarball references in os_list_v3.json file. Until this is done, you should keep both the old and new tarfiles in place, otherwise NOOBS users will not be able to download LibreELEC during this transition period. Once it is all in-sync again, the old tarfiles can be removed.


    As an aside, I have developed an algorithm for shortening partition names appropriately for their file system type in PINN to get around this issue, but it makes some assumptions and is unlikely to get into NOOBS for a while, if at all. But changing the partition names at the source will give a much more predictable outcome.

    Thanks.

    Edited once, last by procount (November 22, 2017 at 11:39 AM).

    OK, so looking at os_list_v3.json:

    os_list_v3.json

    the vast majority of distributions just call their partitions "boot" and "root", and name their tar files accordingly.

    Since this will issue will affect Lakka too, I think calling the partitions (and tar files) simply "System" and "Storage" should work. There seems little point trying to use an "LE" prefix (not sure what Lakka would use, and this has to be codified somehow).

    Ahh, I missed that Lakka has this issue too! Maybe they could use LKSystem and LKStorage?

    There are 17 OSes listed in os_list_v3.json, none of which I have responsibility for. A lot do have just boot and root partition names. However, when you have installed several OSes alongside each other and their partition names are boot, root, boot0, root0, boot1, root1, boot2, root2 etc. it does become difficult to work out which is which!

    I have created noobs images of an additional 30 OSes (See pinn-os/os_all.json at master · procount/pinn-os · GitHub) that can be installed by PINN and mostly by NOOBs. In most cases I have chosen to add the OSname as a suffix to improve this situation, so I provide names like boot_void, root_void, boot_gen, root_gen etc. to easily distinguish them. (Although I too have some partition names that I need to adjust because they are too long as well)

    PINN can also install another 12 OSes from Matt Huisman's repository, which includes Rasplex that also uses System and Storage partition names.

    So if there are several OSes installed with "System" and "Storage" as their partition names, the second one will get renamed to System0 & Storage0, the third to System1 & Storage1 etc.

    So that's my rationale for adding a prefix, but you are free to choose as you wish.

    I will gladly review your PR, but let me know if you're going to stick with your naming or add a prefix, so that I only need to review it once. ;)

    Is the System 0, System1 issue something a user is ever made aware of, or is it something only someone such as yourself gets to see (and is a mostly cosmetic issue)?

    I'm thinking we should cut our loses and stick with a simple no-prefix name for partitions as whatever name we choose we run the risk of a partition name clash, and if we did add a prefix we then need to implement rules for what can be used as a prefix (ie max 3-4 chars, to avoid busting the FAT limit again, and a prefix that doesn't conflict with an existing prefix - and I've no idea how to prevent that!)

    Is it possible to install both RPi1 and RPi2 LibreELEC images on the same NOOBS/PINN SD card, say if a user transports a single SD card between RPi1 and RPi2 devices? If so, even with a prefix, we'd have another unavoidable partition name clash.

    The only remaining solution would be to contrive a unique partition name, eg. lerpi2sys, but then almost no user will understand this (assuming they even need to) and it becomes almost as meaningless as System0.

    So, as long as NOOBS/PINN handles any partition name clash gracefully, then I hope that using the simple, without prefix, partition name as per the PR will suffice. It might not be perfect, but it should work... right? :)

    The System0, System1 issue is just NOOBS' way of making the partition name unique (which it must be) in case of a clash. I guess general users of LibreELEC would not normally see such partition names when using LibreELEC/Kodi, but anyone who can get to a shell can see them with lsblk, mount, fdisk etc. Maybe if you auto-mount additional partitions in Kodi you would see these partition names as the mount points? I don't know (sorry I'm not a big user of media centres).

    Well, even if the prefix clashed, NOOBS/PINN would still add an index to make them unique.

    Yes, it is possible to install both RPi1 and RPi2 versions on the same SD card (for reasons of e.g. creating an SD card for another RPi System), but it is not the default. The user has to add the `showall` option to the cmdline to enable this feature, otherwise it only shows OSes appropriate to the current RPi model it is running on.

    In PINN I'm adding an algorithm to reduce any overly long partition name to fit the file systems. You can see some examples in my initial NOOB issue post here ->LibreELEC labels · Issue #456 · raspberrypi/noobs · GitHub (and a link to a draft of the actual algorithm). Hopefully this contrived shortening will maintain the appropriate parts of the partition name to still be meaningful. I will probably push it as a PR to NOOBS, but it may not get implemented there as I indicated before.

    OK, so noobs/pinn will still work with System and Storage. It solves the immediate issue.

    I shall review your PR as is. Do you want me to just comment or approve? (Not done this process on Git before. Probably I don't have permissions to approve anyway!)

    The tarballs can actually be named differently to the label.

    You just need to include "tarball" in the partitions file.

    This way, you wouldn't need to rename the tarballs, or change the os_list_v3.json file.
    Simply change the partitions.json file by adding the "tarball" and then changing the "label".

    It might be a good idea to do this even if the names are the same (explicit vs implicit).

    eg. The below should work fine

    UPDATE: Only works in PINN due to this pull request not yet merged into NOOBS

    Edited 8 times, last by mattrix (November 23, 2017 at 12:36 AM).

    mattrix, Yes, it's possible when it's a remote system you don't have control over, but I would avoid it unless it is absolutely necessary. (I can't remember if NOOBS works this way too and if it works with the recent download function)

    Well I'm not a NOOBS expert, so I'm having to take direction from this thread at the moment. Adding the tarball filenames into the partitions.json sounds better than relying on the label names which are simply too short to be of much use for anything.

    Quote from procount

    (I can't remember if NOOBS works this way too and if it works with the recent download function)

    Is there any way either of you could confirm that adding the tarball filename into partitions.json will work with NOOBS? This wouldn't change the System/Storage partition labels, but would avoid the external changes resulting from a tarball filename change.

    milhouse - We have investigated and conferred and agreed that you should ignore Mattrix's comments about the tarball parameter, in order to maintain compatibility with NOOBS.

    I've reviewed your Pull Request and it looks ok to me, as far as the partition name changes are concerned. The directory changes look ok as far as I can tell, but I guess you will test it further anyway. If you provide the URLs to the new tarballs I can perform a test by creating a dummy os_list.json file and test it installs ok under NOOBS and PINN before you ask the RPF to update their os_list_v3.json file.

    @milhouse

    Just a note:

    You'll need Raspberry Pi to update their os_list_v3.json with the new tar urls once the change happens.

    May I suggest you keep a copy of the old named files on the servers, so there is no down-time between the files being renamed on LibreELEC's end and Raspberry Pi updating the URLS.

    Once they update their file, then it would be safe to delete the old named files.

    I've uploaded a NOOBS LibreELEC 9.0/Kodi 18a1 test build for RPi2, here: Index of /noobs/LibreELEC_RPi2/

    This includes the shortened partition labels and renamed tar files (does not include the old tar files) - basically everything in NOOBS: shorten partition labels, add $DEVICE support by MilhouseVH · Pull Request #2243 · LibreELEC/LibreELEC.tv · GitHub.

    If anyone can test this it would be appreciated.

    I've got a branch for libreelec-8.2 with essentially the same changes, Comparing LibreELEC:libreelec-8.2...MilhouseVH:le82_fix_noobs · LibreELEC/LibreELEC.tv · GitHub - this branch doesn't require all the extra $DEVICE changes so it's basically just the partition and tar filename changes.