[Orange Pi 3 LTS] Freezes

    i admit i was a little unsure what to do with the update you posted (it wasnt an IMG file that could be burned to an SD card).

    so i took an SD card that i burned with the latest v12 nightly image, mounted it on a linux PC and replaced the SYSTEM and KERNEL files with the files from the "target" folder in the tar you posted. It booted up and was now reporting that it was running your development version. Did I do that correctly?

    Anyway it ran fine and was working, including CEC. But I left it idle though and it froze after about 20 minutes. After a power cycle it it immediately says "[SKIP] Ordering cycle found, skipping local-fs.target" and sits there.

    Quote from jernej

    LE11 test update in the same place.

    Can you push a branch/commit to Github? Or an LE12 aarch64 build? I am already on LE12 aarch64 (so I couldn't use the previous build as it was armhf). I have similar problems with OPi3LTS. However, the results are a bit inconclusive it was stable a few days with the devmem turning off bit 30. CEC did not work for me, not that I care about CEC. The other thing is dmesg is spammed with this (in the unmodified condition):

    dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    I am using it mostly to playback MPEG2 progressive/interlaced (I think it's typically 720p or 1080i MPEG2 for US broadcast, IIRC) recordings & live from Tvheadend. Usually the freezing is after the TV is off a while.

    Quote from jernej

    It's just with this patch: http://ix.io/4FbV I will send it for inclusion in upstream regardless since it makes code aligned with user manual. Regarding CEC, I was never too sure about https://github.com/LibreELEC/Libr…-on-error.patch Can you try to build without it? It gives those CEC warnings in dmesg.

    I based on master, which 6.5 seems to have other issues that may not have been worked out yet (with or without the patches discussed):

    # From boot up/kodi init

    [ 42.389083] cec-dw_hdmi: polling for LA 1 failed with tx_status=0x0030

    [ 42.392601] cec-dw_hdmi: polling for LA 2 failed with tx_status=0x0030

    [ 42.403417] cec-dw_hdmi: polling for LA 9 failed with tx_status=0x0030

    # During playback of an mpeg2 video

    [ 78.112969] sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    [ 80.125761] cedrus 1c0e000.video-codec: frame processing timed out!


    So rebasing on db892cd77 (2023-08-29 nightly, using 6.4.8 -- I think I was using an 8/24 nightly before)


    With both patches, playback works, CEC does not work, and this in dmesg:

    [ 17.740061] cec-dw_hdmi: polling for LA 1 failed with tx_status=0x0030

    [ 17.740639] cec-dw_hdmi: polling for LA 2 failed with tx_status=0x0030

    [ 17.742184] cec-dw_hdmi: polling for LA 9 failed with tx_status=0x0030


    Keeping this patch: https://github.com/LibreELEC/Libr…-on-error.patch, playback works, CEC works, just the usual LOW_DRIVE warnings. I'll run with this build and report back in a few days.

    [ 16.875517] dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    [ 16.983208] dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE


    I'm running Orange Pi 3 LTS from eMMC.

    Edit: It didn't take long, it froze in 15 minutes while watching a video. Oddly the shell was still open, was able to issue dmesg and now it's hung on the shell as well. Usually there is nothing in journalctl (have that set to persistent) and the shell is disconnected.

    I am thinking that playback freeze maybe is related to how unforgiving Allwinner HW decoding is, some of the Tvheadend recordings refused to playback with the following errors:

    [ 78.112969] sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    [ 80.125761] cedrus 1c0e000.video-codec: frame processing timed out!

    I am remembering that mpeg2 decode was a problem perhaps? The same videos playback fine on RPi4, which is I believe ffmpeg software decoding. Anyways I am back to testing that build watching mpeg2 live streams, which is usually just used for background noise while I work.

    Is there a playback freeze or system crash? In any case, MPEG2 is easy to decode, so you can disable MPEG2 HW decoding. If you have a sample, I can take a look, but no promises. I found some workarounds for corrupted H264 streams, but not yet for MPEG2.

    Quote from jernej

    Is there a playback freeze or system crash? In any case, MPEG2 is easy to decode, so you can disable MPEG2 HW decoding. If you have a sample, I can take a look, but no promises. I found some workarounds for corrupted H264 streams, but not yet for MPEG2.

    I think there are two problems here, at least for me, but I am probably one of the few weirdos still using MPEG2 :). I don't transcode the US broadcast, all I do is capture it in a MKV container with the original broadcast streams.

    I was thinking the same thing about disabling HW decode for MPEG2, and I already added a patch to disable MPEG2 decoding capability from being advertised by the cedrus driver and that seems to be working (CPU is running at 15%-20% on all cores when playing MPEG2).

    Three issues were going on for me:

    1. Some MPEG2 videos refused to play and causes Kodi hang ups (eventually it gives up) and timeouts (these are pre-recorded, and I can share clips, but I am waiting to see if this solves problem #3 and then circle back with HW decoding back on to verify I am not missing something):

    [ 78.112969] sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    [ 80.125761] cedrus 1c0e000.video-codec: frame processing timed out!

    2. MPEG2 live tv streams (at least once) caused a playback freeze and system freeze (console was not responsive), nothing on the serial console (which I connected after #3), video playback stopped and audio just stuttered and repeated (probably whatever is in the buffers).

    3. When the device is idle and TV is off the system would freeze, SSH would be disconnected, nothing in the system journal other than ntp adjust.

    Focusing on #3 at the moment, which is similar to the OPs problem. I suspect #1 & #2 are probably the same issue, something it doesn't like about the mpeg2 broadcast streams (perhaps it is corrupted frames or something else).

    When you get frame processing timed out!, system shouldn't freeze/crash. That's why there is iommu, to prevent accessing non-allocated memory and thus corrupting memory. But then again, Cedrus might block one or more memory buses, which freezes the system regardless.

    Disabling CEC doesn't solve issue #3 for you?

    Quote from frakkin64

    [ 42.389083] cec-dw_hdmi: polling for LA 1 failed with tx_status=0x0030

    [ 42.392601] cec-dw_hdmi: polling for LA 2 failed with tx_status=0x0030

    [ 42.403417] cec-dw_hdmi: polling for LA 9 failed with tx_status=0x0030

    Are you testing this on CEC capable device?

    hello, i ve been some days offline, now i see many new messages about freezing issue, just send that devmem command via ssh but not sure if i did right, shoud i expect any feedback message after sending it?, do i need to re-enable CEC in settings?

    Im not sure if my old tv (13 years old or so) is CEC capable, i see an option in settings "HDMI information display" but dont know what is this about, however i set it off days ago with no changes.

    Edited 3 times, last by jordixPI3 (September 3, 2023 at 10:54 AM).

    Quote from jernej

    When you get frame processing timed out!, system shouldn't freeze/crash. That's why there is iommu, to prevent accessing non-allocated memory and thus corrupting memory. But then again, Cedrus might block one or more memory buses, which freezes the system regardless.

    Correct, initially you get the spinning disc in Kodi, the video playback does not start, the player is never launched into the foreground. You can go into the player, stop, and then if you try to play again then things typically get hosed up. If you leave it alone, eventually Kodi seems to timeout.

    Quote from jernej

    Disabling CEC doesn't solve issue #3 for you?

    By the time I came across this thread I went straight to the devmem command, which seemed to work.


    Quote from jernej

    Are you testing this on CEC capable device?

    Yes, it's an older (pre smart tv/web os) LG model with Simplink (CEC). Normally I use an USB remote for Kodi, and that will power on the TV, but I can also use the TV remotes play/pause/stop and arrow keys to navigate Kodi. This error was on 6.5, and also on 6.4.8 when I dropped patch #24 (CEC didn't work without it), with patch #24 CEC is working fine.

    So far 15 hours uptime, and no freeze.

    Any of you have serial console? I plan to build LE11 update with a lot of kernel debug functionality enabled. However, in order to be useful, it must output issues on serial in real time, so hopefully data is received before lock up.