S805x build (lafrite)

    There is a whole lot of stuff about these three systems but none of it makes simple comparison between them let alone explain them well. I tried to make sense of it but my head exploded. Then I thought I had got it so I tried writing it down and my head exploded again. (see summary of implementations) I thought it will be useful to post this here as there were many stackexchange questions asking about pairwise comparisons between them.

    Summary of what to use when:

    1. ZRAM if you have no swap device on HDD/SSD.
    2. ZSWAP if you do have a swap device on HDD/SSD.
    3. ZCACHE: It does what ZSWAP does and ALSO compresses and speeds the filesystem page cache. (It is internally much more complicated and is not in the mainline kernel as it is still under development).

    Summary of their implementations:

    1. ZRAM is a compressed RAM based swap device
    2. ZSWAP is a compressed Cache if you already have a swap.
    3. ZCache is a backend for a special type of Virtual RAM thingy (Transcendent memory) that can be used to cache filesystem pages or swap data.

    Details:

    • ZRAM: Makes a swap device in the RAM. Pages sent here are compressed as they are stored. It has a higher priority than other swap devices: pages that are swapped out are preferentially sent to the zram device till it is full, only then are any other swap devices used.
      • Benefits: Independent of other (physical) swap devices. It can be used when there is no swap partition to expand the available memory.
      • Disadvantages: If other swap devices (HDD/SSD) are present they are not used optimally. As the zram device is an independent swap device, once it is full, any new pages that need to be swapped out are sent to next swap device directly, hence:
        1. There is a real chance of LRU (least recently used) inversion: It will be the most recently swapped data that goes to the slow disk, while inactive pages that were swapped out long ago will remain in the fast ZRAM
        2. The data sent to and read from the disk will consume a lot of bandwidth as it is uncompressed.
      • Status: Merged into the mainline kernel 3.14. Once enabled on a system, it requires some userspace configuration to set up the swap devices and use them.
    • ZSWAP: The frontswap system hooks attempts to swap out pages and uses zswap as write-back-cache for a HDD/SSD swap device: An attempt is made to compress the page and if it contains poorly compressible data it is directly written to the disk. If the data is compressed, it is stored in the pool of zswap memory. If pages are swapped out of memory when the total compressed pages in RAM exceeds a certain size, the Least Recently Used (LRU) compressed page is written to the disk as it is unlikely to be required soon.
      • Benefits: Very efficient use RAM and disk based swap. Minimizes Disk I/O by both reducing the number of writes and reads required (data is compressed and held in RAM) and by reducing the bandwidth of these I/O operations as the data is in a compressed form.
      • Limitations: It is an enhancement of disk based swap systems and hence depends on a swap partition on the hard disk.
      • Status: Merged into the 3.11 mainline linux kernel.
    • ZCache: It is a backend for the Transcendent memory system. Transcendent memory provides a RAM-like memory that can only be accessed a page at a time by using put and get calls. This is unlike normal memory that can be accessed a byte at a time. The frontswap and cleancache systems hook attempts to swap and reclaim file-system page caches respectively and send them to the transcendent memory backends. When zcache is used as a backend, the data is compressed and stored in the RAM. When it fills up, compressed pages are evicted to the swap. (an alternate backend is RAMster which shares a pool of RAM across networked computers). Using only the frontswap frontend with the zcache backend works just like zswap. (In fact zswap is a simplified subset of zcache)
      • Benefits Provides compressed caching both for swap and for filesystem caches.
      • Status: Still not mainlined as it is very complicated and is being worked on.

    I think enabling zRAM will be enought solution for libreelec. Otherwise if it is possible to enable zRAM with in pair with zSWAP with swapfile on disk i think this will also be solution /more complex but if it's only possible /zRAM+zSWAP....

    Edited 2 times, last by stpf99 (March 29, 2020 at 12:54 PM).

    • Official Post

    If you set defaults to -Os it will optimise the image for file size not "running in RAM size" which is the issue. LE sets -O2 which is sensible.

    And .. every couple of years some user discovers this wonderconfig they read about called zram, makes a lot of noise, and rushes off to do extensive performance testing that proves zram has negligible/zero benefit in our distro. Boards under 1GB don't have enough RAM to really benefit from loading more things into RAM and 1GB+ we already run everything in a ramdisk so forcing zram use only adds additional overhead.

    • Official Post

    LibreELEC.tv/linux.aarch64.conf at amlogic-master · chewitt/LibreELEC.tv · GitHub

    ^ If I change the CMA fall-back value from 256 > 128 and force a 1GB device to 512M via kernel boot params, the Kodi GUI comes up and I can play the 1080p60 version of Big Buck Bunny (test media). I need to run some experiments on a wider variety of hardware and codecs before I can consider making that value the distro default. It's possible that HEVC (which is not supported yet) might require more. Let me know how that works for you?

    0.git clone --depth=1 GitHub - chewitt/LibreELEC.tv: Just enough OS for KODI --branch amlogic-master

    cd ~/LibreELEC.tv

    1.PROJECT=Amlogic DEVICE=AMLGX ARCH=arm tools/download-tool

    2.rm packages/linux/patches/amlogic/amlogic-0081-HACK-set-meson-gx-cma-pool-to-896MB.patch

    3.Edit LibreELEC.tv/linux.aarch64.conf at amlogic-master · chewitt/LibreELEC.tv · GitHub

    change locally the CMA fall-back value from 256 > 128 in /projects/Amlogic/linux/linux.aarch64.conf (line 5817)

    apt install docker docker-compose

    docker build --pull -t libreelec tools/docker/bionic

    docker run -v ~/:/home/docker -h libreelec -it libreelec

    cd ~/LibreELEC.tv
    5.PROJECT=Amlogic DEVICE=AMLGX ARCH=arm UBOOT_SYSTEM=lafrite make image

    i will be building for "mali" lafrite 512mb this time for more stable experience / this changed also: projects/Amlogic/devices/AMLGX/options

    + zram as module

    ################################################################################

    # setup device defaults

    ################################################################################

    # The TARGET_CPU variable controls which processor should be targeted for

    # generated code.

    case $TARGET_ARCH in

    aarch64)

    TARGET_CPU="cortex-a53"

    TARGET_CPU_FLAGS="+crc+fp+simd"

    TARGET_FEATURES="64bit"

    ;;

    arm)

    TARGET_KERNEL_ARCH="arm64"

    TARGET_PATCH_ARCH="aarch64"

    TARGET_FLOAT=hard

    TARGET_CPU="cortex-a53"

    TARGET_CPU_FLAGS="+crc"

    TARGET_FPU="neon-fp-armv8"

    TARGET_FEATURES="32bit"

    ;;

    esac

    # OpenGL-ES implementation to use

    OPENGLES="libmali"

    # Graphic drivers to use

    GRAPHIC_DRIVERS="mali"

    # KODI Player implementation to use

    KODIPLAYER_DRIVER="$OPENGLES"

    # Mali GPU family

    MALI_FAMILY="450"


    autostart.sh

    #!/bin/bash

    (

    modprobe zram

    swapoff -a

    totalmem=`free | grep -e "^Mem:" | awk '{print $2}'`

    mem=$(( $totalmem * 512))

    echo $mem > /sys/block/zram0/disksize

    mkswap /dev/zram0

    swapon -p 100 /dev/zram0

    ) &

    Edited 6 times, last by stpf99 (April 1, 2020 at 8:39 PM).

    I have one error when compillling image , package mali-utgard fails on buildnig for mali 450 , found fix for 5.6 about previous error but not for this...

    fix for mali-utgart 5.6-rc6 @file mali_osc_time.c

    Edited 6 times, last by stpf99 (April 3, 2020 at 4:57 PM).

    Thank's for Your help chewitt !

    LaFrite 512mb finally booting:

    LaFrite Mali – Google Drive

    build with:

    mali+ kernel 5.6.0-rc6 /zram as module included

    and memory changes /parametr CMA 256>128

    h264 sd ok /no deinterlacer/ change to egl solve problem

    h264> ok

    hevc4k> not ok / enable zram might help/ not tested

    hevc1080p >not tested

    lan > ok

    builtin audio hdmi out > ok


    for enable zram

    autostart.sh put to /storage/.config/

    >>>>>>>>>>>>>>>>>>>

    #!/bin/bash

    (

    modprobe zram

    swapoff -a

    totalmem=`free | grep -e "^Mem:" | awk '{print $2}'`

    mem=$(( $totalmem * 512))

    echo $mem > /sys/block/zram0/disksize

    mkswap /dev/zram0

    swapon -p 100 /dev/zram0

    ) &

    >>>>>>>>>>>>>>>>>>>

    for stability issues Edit extlinux/extlinux.conf by simply delete additional boot parametrs /not neccesary but error related


    Code
    LABEL LibreELEC

LINUX /KERNEL

FDT /meson-gxl-s805x-libretech-ac.dtb

APPEND boot=LABEL=LIBREELEC disk=LABEL=STORAGE

    Edited 3 times, last by stpf99 (April 5, 2020 at 11:45 AM).

    1720x720p tos-1720x720-cfg01.mkv ok in sw

    1080p jellyfish-3-mbps-hd-hevc.mkv not ok

    4K tos-4096x1720-tiles.mkv crash

    i try to incrase cma to 160mb 128+32

    How to disable loading Libreelec to RAM as ramdisk, it is possible without recompiling??? what to change??

    LibreELEC:~ # df -kh

    Filesystem Size Used Available Use% Mounted on

    devtmpfs 145.4M 4.0K 145.4M 0% /dev

    /dev/mmcblk0p1 511.7M 104.6M 407.1M 20% /flash

    /dev/loop0 95.0M 95.0M 0 100% /

    /dev/mmcblk0p2 13.7G 352.3M 13.4G 3% /storage

    tmpfs 212.5M 0 212.5M 0% /dev/shm

    tmpfs 212.5M 7.5M 205.1M 4% /run

    tmpfs 212.5M 0 212.5M 0% /sys/fs/cgroup

    tmpfs 212.5M 2.5M 210.0M 1% /var

    tmpfs 212.5M 0 212.5M 0% /tmp

    LibreELEC:~ # blkid

    /dev/mmcblk0p1: SEC_TYPE="msdos" LABEL_FATBOOT="LIBREELEC" LABEL="LIBREELEC" UUID="0204-3742" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="ad19b95a-01"

    /dev/mmcblk0p2: LABEL="STORAGE" UUID="351f72ac-2178-41f8-a206-1678ffac25d1" BLOCK_SIZE="1024" TYPE="ext4" PARTUUID="ad19b95a-02"

    /dev/loop0: TYPE="squashfs"


    LibreELEC:~ # free

    total used free shared buff/cache available

    Mem: 435292 261904 47464 4240 125924 114876

    Swap: 217644 9472 208172

    LibreELEC:~ # swapon

    NAME TYPE SIZE USED PRIO

    /dev/zram0 partition 212.6M 9.3M 100

    LibreELEC:~ # cat /proc/mounts

    devtmpfs /dev devtmpfs rw,relatime,size=148908k,nr_inodes=37227,mode=755 0 0

    proc /proc proc rw,relatime 0 0

    sysfs /sys sysfs rw,relatime 0 0

    /dev/mmcblk0p1 /flash vfat ro,noatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro 0 0

    /dev/loop0 / squashfs ro,relatime 0 0

    /dev/mmcblk0p2 /storage ext4 rw,noatime 0 0

    tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0

    devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0

    tmpfs /run tmpfs rw,nosuid,nodev,mode=755 0 0

    tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0

    cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0

    cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0

    none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0

    cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0

    cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0

    cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0

    cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0

    cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0

    cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0

    cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0

    cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0

    cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0

    tmpfs /var tmpfs rw,relatime 0 0

    mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0

    debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0

    tmpfs /tmp tmpfs rw,nosuid,nodev 0 0

    configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0

    LibreELEC:~ #

    grep zram /proc/mounts

    hm nothing here


    is it posible to rmount all tmps mounts to zram pool ?

    i found "systemd-remount-fs.service" but i don't know how to invoke

    to be remounted to zram all non physical mount points ???

    Edited 8 times, last by stpf99 (April 3, 2020 at 6:47 PM).

    pivot_root: change the root filesystem - Linux Man Pages (8)

    i'm trying to use pivot_root for squashfs mounted on /dev/loop0 to recover ram amount but:

    LibreELEC:~/new_root # pivot_root . /

    -sh: pivot_root: not found

    it is in package util-linux but is not anabled in package.mk by default in current amlogic-master

    i don't know also is this command is correct

    Code
    mkdir /storage/root
mount /dev/loop0 /storage/root
cd /storage/root
pivot_root . /
exec chroot . sh <dev/console >dev/console 2>&1
umount /dev/loop0

    Edited 5 times, last by stpf99 (April 3, 2020 at 5:33 PM).

    For devices with low ram </=512 but with EMMC storage LibreElec should have available some tunning options included:

    1.CMA value if possible should be configurable by config file if 128mb is not enought / 256mb is too much.

    but manipulating this value is probably not neccesary if:

    2.should have option to disable devtmpfs tmpfs creation in ram ,systemd service exist for that but must be preconfigured .

    3 have option to image be on ram /dev/loop0 or unsquashed on emmc storage by pivot_root /need to be enabled/included (linux-utils) before compilaton.

    4. zram with higher prio zswap with lower prio enabled

    this will recover about:

    devtmpfs >>145.4M<< 4.0K 145.4M 0% /dev

    /dev/loop0 >>95.0M<< 95.0M 0 100% /

    {

    tmpfs 212.5M 0 212.5M 0% /dev/shm

    tmpfs 212.5M 7.5M 205.1M 4% /run

    tmpfs 212.5M 0 212.5M 0% /sys/fs/cgroup

    tmpfs 212.5M 2.5M 210.0M 1% /var

    tmpfs 212.5M 0 212.5M 0% /tmp

    }

    >>212.5M<<

    145.4 + 95 + 212.5 = 453 M ram / reserved but with idle use

    + system will have more for decoding 4k hevc format

    - system will be slower in operation /emmc spped dependent

    whole infrastructure not need to be reingenered , this proposition is something like "tuning options" and user should have option to enable/disable above tunning steps...

    • Official Post

    Our normal approach is to encourage a couple of capable users to create some unofficial community images that are published/shared here in the forums. Set "BUILDER_NAME=stpf99" when you build images and this is embedded and we can see exact stats on the number of users the images attract. If things ever get popular we can move to "adopt" the changes. If not, we don't have to add complexity for a niche audience.

    CoreELEC/linux.aarch64.conf at 1d6b09df32f1714645eab02ca072ef8b38918b07 · CoreELEC/CoreELEC · GitHub CMA In Coreelec is only 8

    i have also some emmc errors in u-boot, maybe it's about this voltage in dts and it's damged in some places

    linux/meson-gxl-s805x-libretech-ac.dts at master · torvalds/linux · GitHub 1.8v

    linux/meson-gxl-s905x-libretech-cc.dts at master · torvalds/linux · GitHub 1.8v

    but previously it is 3.3v [1/2] arm64: dts: meson: libretech-cc: set eMMC as removable - Patchwork

    Edited once, last by stpf99 (April 7, 2020 at 11:44 AM).