Posts by frakkin64

  • [Orange Pi 3 LTS] Freezes

    My last test was to try and repeat what OP reported, turning off CEC from the Kodi side. And the device still froze, so my feeling is CEC is not the cause.

    Interesting, this was on the serial console:

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    Can you give me your exact messages? I'm interested in numbers in them.

    Generally this is what I see.

    Sep 05 13:51:00 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:01 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:01 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:02 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVESep 05 13:51:02 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:02 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:03 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:03 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:17 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:18 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:18 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    Sep 05 13:51:18 LibreELEC kernel: dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    frakkin64 can you confirm, that with new CEC patches, you don't see messages such as:

    cec-dw_hdmi: message 10 timed out

    dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE


    It doesn't matter if CEC on TV is enabled or not.

    If CEC is on from the device side, yes. If CEC is off from the device side, no. But it's not frequent, usually a few times at start up, and then throughout usage.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    That's not that surprising. As I said, CEC and Cedrus doesn't have anything in common, except one shared clock.

    One thing I tried was turning off CEC from the TV side, and well that really pissed off the device (5-6 LOW_DRIVE warnings every 6 seconds) and it actually froze during menu navigation pretty quickly. I'm testing now with CEC turned off on the Kodi side to see if that has any actual influence as reported earlier. I feel like I have already been there, the next thing will be CEC off on both ends.

    Whatever the cause it is locking up everything (USB keyboard, serial console, network, etc).

  • [Orange Pi 3 LTS] MPEG2 HW decoding results in frame processing timed out & page faults

    Quote from jernej

    But that doesn't solve it. Video stream is corrupted. Not sure if it's possible to find a way to normally decode frames. Still, Kodi should deal with corrupted frames better...

    I don't know much about V4L2, but it sounds like the primary focus (in this scenario) is HW decoding video frames and presenting that to the display framebuffer, and I presume something else is synchronizing audio/video and that it should check for corrupted frames. I assume for SW decoding (which works fine) it is just presenting raw video frames to V4L2 (since it is already decoded).

    I would say that this problem isn't unique to Allwinner, as I mentioned Amlogic had the similar struggles with corrupted videos. RPi4 is, of course, using FFmpeg SW decoding and that works fine on all platforms (FFmpeg seems pretty forgiving and does fixups during playback).

    I can wait and see how LE 12 progresses, but I want to say this was an issue with LE 11 because I stopped using this device, went back to the Amlogic device on CE (but it's starting to get a little flakey with the SD card slot) for a while. Maybe I'll give LE11 a test on an SD card, and report back. As it stands now, the workaround me is to just drop MPEG2 in the driver capabilities, and it takes about 50% CPU on all cores for a full 1080i video which is not a big deal.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    Sorry, I mixed up with other methods.

    No worries, I posted a sample over on this other thread for the MPEG2 HW decode issue. In the meantime, I turned off CEC from the TV-side to see how that behaves and I bumped to 6.5 (which is fine, had multiple problems earlier and was confused at first about what was the cause of the MPEG2 issue).

    I don't personally know if CEC is the cause or not, and I am not totally convinced the devmem command helped because I think it did lock up and that's why I asked about the patch you were testing.

  • [Orange Pi 3 LTS] MPEG2 HW decoding results in frame processing timed out & page faults

    With MPEG2 HW decoding enabled on LE 12 (nightly, 08/29) certain MPEG2 videos result in the following errors from the kernel:

    Sep 04 07:02:37 LibreELEC kernel: sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    Sep 04 07:02:39 LibreELEC kernel: cedrus 1c0e000.video-codec: frame processing timed out!

    And in Kodi the experience is a spinning disc while the player is trying to start the video. If you use CEC remote it will result in a Kodi lock-up (but shell is still responsive), if you try to stop and restart the video it will also result in a Kodi lock-up. I tried doing a video clip for uploading with:

    ffmpeg -ss 00:00:00 -to 00:01:00 -i mpeg2-h6-decode-full.mkv -c copy -map 0 mpeg2-h6-decode.mkv

    And it result in these warnings in ffmpeg and a working video in Kodi (original video still does not work and is 2GB):

    [matroska @ 0x556df9223f40] Non-monotonous DTS in output stream 0:0; previous: 2018, current: 1951; changing to 2018. This may result in incorrect timestamps in the output file.

    [matroska @ 0x556df9223f40] Non-monotonous DTS in output stream 0:0; previous: 2018, current: 1985; changing to 2018. This may result in incorrect timestamps in the output file.

    Here is the sample, copied with dd partially, that results in this issue:

    mpeg2-h6-decode.mkv

    Thanks chewitt for the tip. :)

  • [Orange Pi 3 LTS] Freezes

    jernej It finally locked up. The shell via ssh was disconnected, and the serial console has nothing on it and is non-responsive.

    Quote from chewitt

    AMLGX includes this for a while now: https://github.com/chewitt/linux/…23a7a19f2d0ba9e

    Yep, I remember doing something similar as well on AMLGX. I guess in a purist view the HW decoding should work, so I can see why Kodi & Linux don't have a knob to turn it off from a configuration perspective. :) Hopefully we will get there eventually, but MPEG2 is not a big burden to SW decode for most CPUs now

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    Can't you specifically disable MPEG2 HW accel in settings (maybe expert view is needed)?

    I have never seen such an option with DRMPRIME. I think that is available with VA API? If it's possible would like to hear any tips about how to do that, didn't see anything in Player or System and I am at expert level, it would be a useful feature as Amlogic mainline also has MPEG2 HW decoding problems.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    what kind of changes do you have?

    Mainly just these two patches (the one you sent, and one I did to disable MPEG2_DEC capability in cedrus on h6):

    patches · LibreELEC/LibreELEC.tv@e7f58c9
    github.com
    allwinner: disable mpeg2 on h6 devices · LibreELEC/LibreELEC.tv@a95af0a
    github.com

    It's based on the 8/29 nightly with two other patches cherry-picked to resolve the dependency issue with systemd.

    Comparing LibreELEC:master...wagnerch:opi3lts-b3-sw · LibreELEC/LibreELEC.tv (github.com)

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    Any of you have serial console? I plan to build LE11 update with a lot of kernel debug functionality enabled. However, in order to be useful, it must output issues on serial in real time, so hopefully data is received before lock up.

    I have a serial console enabled, but I am on LE12 aarch64 builds. If you share the patch then I can build with it.

    So far, after 22 hours of uptime there has been no freezing -- it is a mix of TV off/idle (which is when it usually freezes), MPEG2 live TV streams, and H264 encoded TV shows.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    When you get frame processing timed out!, system shouldn't freeze/crash. That's why there is iommu, to prevent accessing non-allocated memory and thus corrupting memory. But then again, Cedrus might block one or more memory buses, which freezes the system regardless.

    Correct, initially you get the spinning disc in Kodi, the video playback does not start, the player is never launched into the foreground. You can go into the player, stop, and then if you try to play again then things typically get hosed up. If you leave it alone, eventually Kodi seems to timeout.

    Quote from jernej

    Disabling CEC doesn't solve issue #3 for you?

    By the time I came across this thread I went straight to the devmem command, which seemed to work.


    Quote from jernej

    Are you testing this on CEC capable device?

    Yes, it's an older (pre smart tv/web os) LG model with Simplink (CEC). Normally I use an USB remote for Kodi, and that will power on the TV, but I can also use the TV remotes play/pause/stop and arrow keys to navigate Kodi. This error was on 6.5, and also on 6.4.8 when I dropped patch #24 (CEC didn't work without it), with patch #24 CEC is working fine.

    So far 15 hours uptime, and no freeze.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    Is there a playback freeze or system crash? In any case, MPEG2 is easy to decode, so you can disable MPEG2 HW decoding. If you have a sample, I can take a look, but no promises. I found some workarounds for corrupted H264 streams, but not yet for MPEG2.

    I think there are two problems here, at least for me, but I am probably one of the few weirdos still using MPEG2 :). I don't transcode the US broadcast, all I do is capture it in a MKV container with the original broadcast streams.

    I was thinking the same thing about disabling HW decode for MPEG2, and I already added a patch to disable MPEG2 decoding capability from being advertised by the cedrus driver and that seems to be working (CPU is running at 15%-20% on all cores when playing MPEG2).

    Three issues were going on for me:

    1. Some MPEG2 videos refused to play and causes Kodi hang ups (eventually it gives up) and timeouts (these are pre-recorded, and I can share clips, but I am waiting to see if this solves problem #3 and then circle back with HW decoding back on to verify I am not missing something):

    [ 78.112969] sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    [ 80.125761] cedrus 1c0e000.video-codec: frame processing timed out!

    2. MPEG2 live tv streams (at least once) caused a playback freeze and system freeze (console was not responsive), nothing on the serial console (which I connected after #3), video playback stopped and audio just stuttered and repeated (probably whatever is in the buffers).

    3. When the device is idle and TV is off the system would freeze, SSH would be disconnected, nothing in the system journal other than ntp adjust.

    Focusing on #3 at the moment, which is similar to the OPs problem. I suspect #1 & #2 are probably the same issue, something it doesn't like about the mpeg2 broadcast streams (perhaps it is corrupted frames or something else).

  • [Orange Pi 3 LTS] Freezes

    I am thinking that playback freeze maybe is related to how unforgiving Allwinner HW decoding is, some of the Tvheadend recordings refused to playback with the following errors:

    [ 78.112969] sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    [ 80.125761] cedrus 1c0e000.video-codec: frame processing timed out!

    I am remembering that mpeg2 decode was a problem perhaps? The same videos playback fine on RPi4, which is I believe ffmpeg software decoding. Anyways I am back to testing that build watching mpeg2 live streams, which is usually just used for background noise while I work.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    It's just with this patch: http://ix.io/4FbV I will send it for inclusion in upstream regardless since it makes code aligned with user manual. Regarding CEC, I was never too sure about https://github.com/LibreELEC/Libr…-on-error.patch Can you try to build without it? It gives those CEC warnings in dmesg.

    I based on master, which 6.5 seems to have other issues that may not have been worked out yet (with or without the patches discussed):

    # From boot up/kodi init

    [ 42.389083] cec-dw_hdmi: polling for LA 1 failed with tx_status=0x0030

    [ 42.392601] cec-dw_hdmi: polling for LA 2 failed with tx_status=0x0030

    [ 42.403417] cec-dw_hdmi: polling for LA 9 failed with tx_status=0x0030

    # During playback of an mpeg2 video

    [ 78.112969] sun50i-iommu 30f0000.iommu: Page fault for 0x0000000000007000 (master 1, dir rd)

    [ 80.125761] cedrus 1c0e000.video-codec: frame processing timed out!


    So rebasing on db892cd77 (2023-08-29 nightly, using 6.4.8 -- I think I was using an 8/24 nightly before)


    With both patches, playback works, CEC does not work, and this in dmesg:

    [ 17.740061] cec-dw_hdmi: polling for LA 1 failed with tx_status=0x0030

    [ 17.740639] cec-dw_hdmi: polling for LA 2 failed with tx_status=0x0030

    [ 17.742184] cec-dw_hdmi: polling for LA 9 failed with tx_status=0x0030


    Keeping this patch: https://github.com/LibreELEC/Libr…-on-error.patch, playback works, CEC works, just the usual LOW_DRIVE warnings. I'll run with this build and report back in a few days.

    [ 16.875517] dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    [ 16.983208] dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE


    I'm running Orange Pi 3 LTS from eMMC.

    Edit: It didn't take long, it froze in 15 minutes while watching a video. Oddly the shell was still open, was able to issue dmesg and now it's hung on the shell as well. Usually there is nothing in journalctl (have that set to persistent) and the shell is disconnected.

  • [Orange Pi 3 LTS] Freezes

    Quote from jernej

    LE11 test update in the same place.

    Can you push a branch/commit to Github? Or an LE12 aarch64 build? I am already on LE12 aarch64 (so I couldn't use the previous build as it was armhf). I have similar problems with OPi3LTS. However, the results are a bit inconclusive it was stable a few days with the devmem turning off bit 30. CEC did not work for me, not that I care about CEC. The other thing is dmesg is spammed with this (in the unmodified condition):

    dw_hdmi_cec_hardirq: stat=11 LOW_DRIVE

    I am using it mostly to playback MPEG2 progressive/interlaced (I think it's typically 720p or 1080i MPEG2 for US broadcast, IIRC) recordings & live from Tvheadend. Usually the freezing is after the TV is off a while.