[x86-64] ZFS Support

dikov · February 26, 2025 at 4:36 AM

Hi there,

I was curious lack of support of ZFS in LibreElec (is there any reason for that?) so i want to build my own.
Three days and still nothing.

Here is my package.md

Code

PKG_NAME="zfs" 
PKG_VERSION="2.3.0" 
PKG_SHA256="6e8787eab55f24c6b9c317f3fe9b0da9a665eb34c31df88ff368d9a92e9356a6" 
PKG_URL="https://github.com/openzfs/zfs/releases/download/zfs-$%7BPKG_VERSION%7D/zfs-$%7BPKG_VERSION%7D.tar.gz" 
PKG_DEPENDS_TARGET="toolchain linux" 
PKG_DESCRIPTION="ZFS filesystem support" 
PKG_TOOLCHAIN="autotools" 

pre_configure_target() { 
cd $PKG_BUILD 
if [ ! -f ./configure ]; then 
echo "configure script missing after autoreconf!" 
exit 1 
fi 
} 

configure_target() { 
cd $PKG_BUILD 
./configure --host=$TARGET_NAME --prefix=/usr \ 
--with-linux=$LINUX_DIR \ 
--with-linux-obj=$LINUX_DIR \ 
--disable-dependency-tracking 
# --disable-code-coverage 
} 

make_target() { 
make 
} 

makeinstall_target() { 
make install DESTDIR=$INSTALL 
} 

Which gives me error

Display More

config.status: creating module/Makefile
config.status: creating rpm/generic/zfs-dkms.spec
config.status: creating rpm/generic/zfs-kmod.spec
config.status: creating rpm/generic/zfs.spec
config.status: creating rpm/redhat/zfs-dkms.spec
config.status: creating rpm/redhat/zfs-kmod.spec
config.status: creating rpm/redhat/zfs.spec
config.status: creating tests/zfs-tests/tests/Makefile
config.status: creating zfs.release
config.status: creating zfs_config.h
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands
make[1]: Entering directory '/home/work/SOFT/Linux/libreElec/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/b
uild/zfs-2.3.0'
Makefile:14040: *** missing separator. Stop.
make[1]: Leaving directory '/home/work/SOFT/Linux/libreElec/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/bu
ild/zfs-2.3.0'
FAILURE: scripts/build zfs:target during make_target (package.mk)
*********** FAILED COMMAND ***********
make
**************************************
*********** FAILED COMMAND ***********
${SCRIPTS}/build "${1}" "${PARENT_PKG}"
**************************************
FAILURE: scripts/install zfs:target has failed!

The following log for this failure is available:
/home/work/SOFT/Linux/libreElec/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/.threads/logs/188.log

>>> zfs:target seq 188 >>>
[303/314] [FAIL] install zfs:target

The following log for this failure is available:
/home/work/SOFT/Linux/libreElec/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/.threads/logs/188.log

Parallel build failure - see log for details. Time of failure: Tue Feb 25 22:26:12 CST 2025
make: *** [Makefile:10: image] Error 1

In the zfs Makefile the line 14430 is

@CODE_COVERAGE_RULES@

I tried to disable --disable-dependency-tracking

But still it did not work as you see.

Any ideas?

Cheers,

**chewitt** · February 26, 2025 at 5:33 AM

Quote from dikov

I was curious lack of support of ZFS in LibreElec (is there any reason for that?)

We intentionally omit all forms of software RAID to reduce the scope of what we need to support and maintain. ZFS might interest a small number of people, but it's a small number. The remaining 99.9995% of our userbase has no clue what ZFS is or how to use or benefit from it's features. There's no objection to someone packaging ZFS as an add-on, but it's not for the core distro.

No ideas on the build issue .. not something I personally have time to look at right now, but others might comment.

**heitbaum** · February 26, 2025 at 8:25 AM

Configure is doing strange stuff and not using the target compiler.

build.LibreELEC-Generic.x86_64-13.0-devel/build/zfs-2.3.0/build/kernel_fpu_license/build.log
make: Entering directory '/var/media/DATA/home-rudi/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/build/linux-6.12.15'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-libreelec-linux-gnu-gcc-15.0.1 (GCC) 15.0.1 20250223 (experimental)
You are using: gcc (Ubuntu 14.2.0-4ubuntu2~24.04) 14.2.0
MODPOST /var/media/DATA/home-rudi/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/build/zfs-2.3.0/build/kernel_fpu_license
/Module.symvers
ERROR: modpost: GPL-incompatible module kernel_fpu_license.ko uses GPL-only symbol 'kernel_fpu_begin_mask'
ERROR: modpost: GPL-incompatible module kernel_fpu_license.ko uses GPL-only symbol 'kernel_fpu_end'

==

this will get you quite a bit further. It is not complete or fully working on the make.

Bash

PKG_NAME="zfs" 
PKG_VERSION="2.3.0" 
PKG_SHA256="6e8787eab55f24c6b9c317f3fe9b0da9a665eb34c31df88ff368d9a92e9356a6" 
PKG_URL="https://github.com/openzfs/zfs/releases/download/zfs-${PKG_VERSION}/zfs-${PKG_VERSION}.tar.gz" 
PKG_DESCRIPTION="ZFS filesystem support" 
#PKG_TOOLCHAIN="autotools" 
PKG_IS_KERNEL_PKG="yes"

configure_target() { 
  #https://github.com/openzfs/zfs/pull/16924
  cd $PKG_BUILD 
  export KERNEL_CROSS_COMPILE=${TARGET_KERNEL_PREFIX}
  export KERNEL_ARCH=${TARGET_KERNEL_ARCH}
  ./configure --host=$TARGET_NAME --prefix=/usr \
  --with-linux=$(kernel_path) \
  --with-linux-obj=$(kernel_path)
} 

make_target() { 
  make V=1 \
       ARCH=${TARGET_KERNEL_ARCH} \
       KSRC=$(kernel_path) \
       CROSS_COMPILE=${TARGET_KERNEL_PREFIX}
} 

makeinstall_target() { 
  mkdir -p ${INSTALL}/$(get_full_module_dir)/${PKG_NAME}
    cp *.ko ${INSTALL}/$(get_full_module_dir)/${PKG_NAME}
}

Display More

dikov · February 26, 2025 at 4:15 PM

Quote from chewitt

We intentionally omit all forms of software RAID to reduce the scope of what we need to support and maintain. ZFS might interest a small number of people, but it's a small number. The remaining 99.9995% of our userbase has no clue what ZFS is or how to use or benefit from it's features. There's no objection to someone packaging ZFS as an add-on, but it's not for the core distro.
No ideas on the build issue .. not something I personally have time to look at right now, but others might comment.

What filesystem for a large storage then many people use? Since I am a Linux guy, I need best compatibility with it.

**Da Flex** · February 26, 2025 at 7:02 PM

Quote from dikov

What filesystem for a large storage then many people use? Since I am a Linux guy, I need best compatibility with it.

For compatibility most people use exFAT. That file system is readable on Windows, MacOS and Linux.

dikov · February 28, 2025 at 6:49 AM

Hi,

Thank you for the answers. With variable set

Code

MAKE=gmake
LINUX_DIR=/full/path/to/LibreElec/kernel/LibreELEC.tv/build.LibreELEC-Generic.x86_64-13.0-devel/build/linux-6.12.15
PROJECT=Generic ARCH=x86_64 make image VERBOSE=1

and package.md

Code

PKG_NAME="zfs"
PKG_VERSION="2.3.0"
PKG_SHA256="6e8787eab55f24c6b9c317f3fe9b0da9a665eb34c31df88ff368d9a92e9356a6"
PKG_URL="https://github.com/openzfs/zfs/releases/download/zfs-${PKG_VERSION}/zfs-${PKG_VERSION}.tar.gz"
PKG_DESCRIPTION="ZFS filesystem support"
PKG_IS_KERNEL_PKG="yes"
PKG_DEPENDS_TARGET="toolchain linux libunwind zlib"

configure_target() {
  cd $PKG_BUILD
  KERNEL_DIR=$(kernel_path)
  if [ ! -d "$KERNEL_DIR" ]; then
    echo "Kernel directory not found: $KERNEL_DIR"
    exit 1
  fi
  export KERNEL_CROSS_COMPILE=${TARGET_KERNEL_PREFIX}
  export KERNEL_ARCH=${TARGET_KERNEL_ARCH}
  ./configure --host=$TARGET_NAME --prefix=/usr \
              --with-linux="$KERNEL_DIR" \
              --with-linux-obj="$KERNEL_DIR" \
              --disable-static \
              --enable-shared
}

make_target() {
  make V=1 \
       ARCH=${TARGET_KERNEL_ARCH} \
       KSRC=$(kernel_path) \
       CROSS_COMPILE=${TARGET_KERNEL_PREFIX} \
       LDFLAGS="$LDFLAGS -lunwind -lz"
}

makeinstall_target() {
  # Install kernel modules to standard fs/zfs directory
  mkdir -p ${INSTALL}/$(get_full_module_dir)/fs/zfs
  find $PKG_BUILD/module -name "*.ko" -exec cp {} ${INSTALL}/$(get_full_module_dir)/fs/zfs/ \;
  # Install user-space tools
  make install DESTDIR=$INSTALL
  # Generate module dependencies
  ${KERNEL_CROSS_COMPILE}depmod -a -b ${INSTALL} 6.12.15
}

Display More

I successfully compiled a built , but it appears that this setup overrides all other drivers in /lib/modules/6.2.15/

Code

ls /lib/modules/6.12.15/ 
extra                      modules.dep.bin 
modules.alias              modules.devname 
modules.alias.bin          modules.softdep 
modules.builtin.alias.bin  modules.symbols 
modules.builtin.bin        modules.symbols.bin 
modules.dep                modules.weakdep 
LibreELEC:~ # ls /lib/modules/6.12.15/extra/ 
spl.ko  zfs.ko

and I need to load zfs with

insmod /lib/modules/6.12.15/extra/spl.ko
insmod /lib/modules/6.12.15/extra/zfs.ko
zpool import -f dSTOR_8Tb
zfs set mountpoint=/var/media/dSTOR dSTOR_8Tb
zfs mount dSTOR_8Tb

How to add zfs drivers to the modules in a right manner?

Thank you

**chewitt** · February 28, 2025 at 7:08 AM

Quote from dikov

What filesystem for a large storage then many people use? Since I am a Linux guy, I need best compatibility with it.

LE supports all the main kernel-native filesystems and EXT4 is our "generates no support issues" default. If you want more advanced filesystem options perhaps look at XFS or BTRFS (which also has a userspace tools add-on) but very few people use them and on that basis I can't make a glowing recommendation. If you want or need software RAID or JBOD features you'll need to build a custom LE image with different kernel options set, or perhaps use sky42 community image for encrypted storage which includes the bits of devicemanager plumbing we omit from the main image.

**heitbaum** · February 28, 2025 at 7:53 AM

No need to declare variables before hand. I don’t know what they will do. Just a scripts/build zfs to build the zfs.

Simple makeinstall_target is all that is necessary.

Code

makeinstall_target() {
  mkdir -p ${INSTALL}/$(get_full_module_dir)/fs/${PKG_NAME}
    cp module/*.ko ${INSTALL}/$(get_full_module_dir)/fs/${PKG_NAME}
}

Code

find build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/.libreelec-package
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base/lib
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base/lib/modules
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base/lib/modules/6.12.15
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base/lib/modules/6.12.15/fs
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base/lib/modules/6.12.15/fs/zfs
build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/usr/lib/kernel-overlays/base/lib/modules/6.12.15/fs/zfs/spl.ko

Display More

dikov · February 28, 2025 at 9:33 PM

Heitbaum,

Thank you for the suggestion. This indeed helped to install zfs kernel modules along with others, so when I boot it i have all modules in place, but zfs userspace tools (zpool and )

How to add them?

dikov · February 28, 2025 at 11:30 PM

I just compared build directories of zfs (installing improperly) to nano (installed properly)
in case of nano there are files withing:

$ ls nano-8.3/.x86_64-libreelec-linux-gnu/
config.h config.log config.status configure.lineno doc lib m4 Makefile po src stamp-h1 syntax
but same is empty in zfs folder.
$ ls zfs-2.3.0/.x86_64-libreelec-linux-gnu/

**heitbaum** · March 1, 2025 at 7:12 AM

Quote from dikov

I just compared build directories of zfs (installing improperly) to nano (installed properly)
in case of nano there are files withing:

$ ls nano-8.3/.x86_64-libreelec-linux-gnu/
config.h config.log config.status configure.lineno doc lib m4 Makefile po src stamp-h1 syntax
but same is empty in zfs folder.
$ ls zfs-2.3.0/.x86_64-libreelec-linux-gnu/

Display More

That is because it is building in the builddir for nano. Whereas zfs is building in the PKGDIR. Builddir is the usual for LE packages.

**heitbaum** · March 1, 2025 at 7:35 AM

Quote from dikov

Heitbaum,
Thank you for the suggestion. This indeed helped to install zfs kernel modules along with others, so when I boot it i have all modules in place, but zfs userspace tools (zpool and )
How to add them?

the command seems to install all sorts of things. But you will need to trial and error find the right install command to get the files you want. Otherwise saferemove the files. Otherwise cp is an option.
make install DESTDIR=${INSTALL}

The below will help you work out what is going to end up in the image.
find build.LibreELEC-Generic.x86_64-13.0-devel/install_pkg/zfs-2.3.0/

dikov · March 6, 2025 at 12:54 AM

After trial and error here is what worked for me. I built in Ubuntu.

Code

$ sudo apt-get update
$ sudo apt-get install gcc g++ gperf flex bison gawk help2man ncurses-dev make exuberant-ctags texinfo git bc libexpat-dev libxml2-dev libxml2-utils xsltproc python-dev python3-dev zlib1g-dev libssl-dev unzip autopoint libtool shtool automake pkg-config gettext
$ git clone https://github.com/LibreELEC/LibreELEC.tv.git
$ cd LibreELEC.tv
$ echo "CONFIG_ZFS=y" >> projects/Generic/linux/linux.x86_64.conf
$ echo "ADDITIONAL_PACKAGES+=\" zfs\"" >> projects/Generic/options
$ mkdir -p packages/filesystem/zfs

create instructions to build a zfs module

Code

$ nano packages/filesystem/zfs

Code

PKG_NAME="zfs" 
PKG_VERSION="2.3.0" 
PKG_SHA256="6e8787eab55f24c6b9c317f3fe9b0da9a665eb34c31df88ff368d9a92e9356a6" 
PKG_URL="https://github.com/openzfs/zfs/releases/download/zfs-${PKG_VERSION}/zfs-${PKG_VERSION}.tar.gz" 
PKG_DESCRIPTION="ZFS filesystem support" 
PKG_IS_KERNEL_PKG="yes" 
PKG_DEPENDS_TARGET="toolchain linux libunwind zlib" 

configure_target() { 
 cd $PKG_BUILD 
 KERNEL_DIR=$(kernel_path) 
 if [ ! -d "$KERNEL_DIR" ]; then 
   echo "Kernel directory not found: $KERNEL_DIR" 
   exit 1 
 fi 
 export KERNEL_CROSS_COMPILE=${TARGET_KERNEL_PREFIX} 
 export KERNEL_ARCH=${TARGET_KERNEL_ARCH} 
 ./configure --host=$TARGET_NAME --prefix=/usr \ 
             --with-linux="$KERNEL_DIR" \ 
             --with-linux-obj="$KERNEL_DIR" \ 
             --disable-static \ 
             --enable-shared 
} 

make_target() { 
 make V=1 \ 
      ARCH=${TARGET_KERNEL_ARCH} \ 
      KSRC=$(kernel_path) \ 
      CROSS_COMPILE=${TARGET_KERNEL_PREFIX} \ 
      LDFLAGS="$LDFLAGS -lunwind -lz" 
} 

makeinstall_target() { 
 # Install ZFS kernel modules 
 mkdir -p ${INSTALL}/$(get_full_module_dir)/fs/${PKG_NAME} 
 cp $PKG_BUILD/module/*.ko ${INSTALL}/$(get_full_module_dir)/fs/${PKG_NAME}/ 
 # Install user-space tools manually 
 mkdir -p ${INSTALL}/usr/sbin 
 cp $PKG_BUILD/.libs/zfs ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zpool ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zdb ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zinject ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zstream ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/ztest ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zhack ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zpool_influxdb ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/mount.zfs ${INSTALL}/usr/sbin/ 
 cp $PKG_BUILD/.libs/zfs_ids_to_path ${INSTALL}/usr/sbin/ 
 # Install shared libraries 
 mkdir -p ${INSTALL}/usr/lib 
 cp $PKG_BUILD/.libs/libzfs.so* ${INSTALL}/usr/lib/ 
 cp $PKG_BUILD/.libs/libzfs_core.so* ${INSTALL}/usr/lib/ 
 cp $PKG_BUILD/.libs/libnvpair.so* ${INSTALL}/usr/lib/ 
 cp $PKG_BUILD/.libs/libuutil.so* ${INSTALL}/usr/lib/ 
 cp $PKG_BUILD/.libs/libzpool.so* ${INSTALL}/usr/lib/ 
 # Ensure ZFS loads on boot 
 mkdir -p ${INSTALL}/etc/modules-load.d 
 echo "zfs" > ${INSTALL}/etc/modules-load.d/zfs.conf 
}

Display More

It's time to compile the build!

Code

$ PROJECT=Generic ARCH=x86_64 make image VERBOSE=1

If it did not crash, proceed to installation. Insert installation USB flash drive (here as /dev/sda)

Code

$ cd target
$ gunzip LibreELEC-Generic.x86_64-13.0-devel-20250301013418-f3936e7.tar.img
$ sudo dd if=LibreELEC-Generic.x86_64-13.0-devel-20250301013418-f3936e7.tar of=/dev/sda bs=4M

Install as usual (Insert the flash drive into a computer that you want to run libreelec; Boot from it, Install LibreElec on another drive; Reboot)

At first launch, enable ssh . Then

$ ssh root@<ip.of.your.htpc>

at htpc

Code

$ modprobe zfs
$ zpool import -f <ZFS_8Tb> (i.e. name of your ZFS_drive)
$ zfs set mountpoint=/var/media/ZFS ZFS_8Tb

To make this ZFS drive visible among in the list of filesystems, folders and drives use Add button and point to /var/media/ZFS.

To automate the process so after reboot it will be there:

nano /storage/.config/autostart.sh

Bash

#!/bin/sh 
depmod -a 6.12.15 
modprobe zfs 
zpool import -f ZFS_8Tb 
zfs set mountpoint=/var/media/ZFS ZFS_8Tb

**chewitt** · March 6, 2025 at 5:28 AM

You can (or should be able to) automate mounting with udev. See https://github.com/openzfs/zfs/tree/master/udev/rules.d