Orange Pi 3 - irregular system freezing

I have had a problem with LibreELEC / Kodi installed on the Orange Pi 3 board for a very long time (from the very beginning).

The problem is that the whole system freezes. In this state, the board stops responding to any command, does not respond to the ping, the SSH session is interrupted and the image on the monitor freezes.

The board responds only to the power button and, of course, to the power off. This problem occurs intermittently, usually within one hour of startup.

I needed to find out whether the cause of the described problem is the nature of HW or SW.

First, I tried to replace everything that could be replaced (SD card, cable, power supply). Unfortunately, no change.

Then I tried to install Kodi in Armbian OS. It was unusable (very slow, just an old version of Kodi).

Finally, I installed the Android 7 OS on the SD card and then Kodi. This is a combination you can live with, at least on a small monitor.

I let it run for a week and everything was fine. The problem did not occur. I am aware that this is not entirely sufficient proof that the board is OK. But I only have one piece.

What should I do ?

kodi.log

serial_console.log

Thanks for the answers, but it's sad for me.

So I won't do a memory test yet.

I understand that it will be difficult to find. No crash log because there is no crash ...

However, such an unreliable system is unusable for normal operation.

In my case, the problem may not be related to ffmpeg. I just restarted the system and then I didn't do anything.

Unfortunately, this freezing also occurs when watching IPTV.

Just for the purpose of this testing, I made a new SD card and did not install any add-ons at all (except for the screensaver).

So this is an almost pure "factory" state as when it was first run.

The course of the test was that I enabled the debug log and rebooted.

The screensaver started automatically and the photos show that the freeze occurred at 06:50, while the last entry in the kodi.log was at 06:32:37.

I was surprised that the serial console log contains only the system boot and then nothing.

Would it be possible to make some debug image that would write in detail to the serial console?

Quote from jernej

no need, just remove quiet from extlinux.conf. If you want terminal on serial, also add systemd.debug-shell=ttyS0

I had the quiet parameter removed before. Now I have also added systemd.debug-shell=ttyS0, so the serial console is fully functional.

So now I can use a serial console to communicate with the LE in a similar way as via an SSH connection.

Once it freezeed, I found that the system was not responding to the serial console either. Like when it's off.

Quote from Da Flex

From what we know until now, it won't hurt to do the test.

That's right, a memory test can't hurt. So I prepared an SD card with the Debian operating system. I found only one software for memory testing, a memtester.

I did some tests, but all the results were OK. Debian worked well, there was no freezing.

Then I did the same and with the same result with the Ubuntu operating system.

So there is no indication that my board is defective.

Nevertheless, I decided to buy another piece of the same board to see if the board was defective or not.

I am aware that in one case I will have two good boards, which, however, cannot be used with LibreELEC ...

I have made considerable progress !

It was like this: At first I made one desperate attempt. I took a new SD card and wrote the LE image for the Orange Pi One Plus on it.

I inserted the card into the Orange Pi 3 and turned on the power. LE and Kodi started normally. I made some settings and installed (among other things) the PVR IPTV Simple Client add-on and my IPTV add-on. I used it for a few days, waiting for a total freeze to occur again. It didn't happen!

So it looks like the freeze problem is probably related to either USB3 or WiFi / BT. To rule out at least one of these, I decided to run at least USB3 (the absence of USB3 bothered me much more than the absence of WiFi and BT). So I found the source file sun50i-h6-orangepi-one-plus.dts and tried to compile it into sun50i-h6-orangepi-one-plus.dtb. But I couldn't. I was still getting the error message "FATAL ERROR: Unable to parse input tree".

So I did it differently. I took the sun50i-h6-orangepi-one-plus.dtb file located in the /flash folder and converted it to the sun50i-h6-orangepi-one-plus.dts file. There I enabled USB3 and did the conversion back to sun50i-h6-orangepi-one-plus.dtb.

I replaced the original file in the /flash folder and turned on the device. It worked - USB3 started working.

I used the device with the modified DT file again for a few days - it never froze!

Then I took the next step. I returned to the original SD card I had earlier in Orange Pi 3. There I replaced the original sun50i-h6-orangepi-3.dtb file with the modified sun50i-h6-orangepi-one-plus.dtb file. After switching on the device, the expected is confirmed: no total freezing.

Result: It seems that the total freezing of the Orange Pi 3 is related to either WiFi or BT. However, it is not enough if I use the original file sun50i-h6-orangepi-3.dtb and disable WiFi and BT in the settings of the LibreELEC add-on.

How do I proceed to find the real cause of the freeze?

Note: I'm still talking about "total freezing". So I say a state where the board did not respond to anything (except turning off the power or pressing the power button). This never happened while testing the device with a modified DT file. However, sometimes (always during IPTV playback) there was a temporary freeze (the image stopped and after a while Kodi restarted spontaneously). This is probably related to the mentioned ffmpeg. Interestingly, this crash never occurred on an Orange Pi 3 board with the original sun50i-h6-orangepi-3.dtb file.

Quote from jernej

LuRu I think I found important difference. While it's true that wireless power supply is a mess, really important bit is CPU voltage range. I copied over One Plus values and my OPi3 seems to be much more stable. Please test this update:

Index of /test/opi3-stability/

That sounds promising!

As soon as I get home today, I'll try it right away. If it's okay, I'll get back to you in a few days. If it's not okay, I'll see it today or tomorrow at the latest.

BTW: I've done some attempts in the meantime (especially on the weekend), but I haven't been successful.

Of course it's not certain yet (freezing is accidental), but it looks good!

It has only one flaw. This is LE11 and updating or installing LE add-ons fails.

How can I compile LE10 with the necessary changes?

Note: I am able to perform a normal compilation (eg with git checkout 10.0.0).

I can't compile it.

I created a 0021-arm64-dts-allwinner-h6-OPi3-stability.patch file to which I copied http://ix.io/3yMQ.

I placed this file in the location you specified.

Compilation failed:

APPLY PATCH (device)      projects/Allwinner/devices/H6/patches/linux/0021-arm64-dts-allwinner-h6-OPi3-stability.patch
patching file arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-3.dts
Hunk #1 succeeded at 73 with fuzz 2 (offset 10 lines).
Hunk #2 succeeded at 155 (offset 39 lines).
Hunk #3 FAILED at 152.
Hunk #4 succeeded at 250 (offset 36 lines).
Hunk #5 succeeded at 265 (offset 36 lines).
1 out of 5 hunks FAILED -- saving rejects to file arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-3.dts.rej
*********** FAILED COMMAND ***********
patch -d $(echo "${PKG_BUILD}" | cut -f1 -d\ ) -p1 1>&${VERBOSE_OUT}
**************************************
*********** FAILED COMMAND ***********
${SCRIPTS}/unpack "${PKG_NAME}" "${PARENT_PKG}"
**************************************
FAILURE: scripts/build linux:host has failed!

Contents of the sun50i-h6-orangepi-3.dts.rej file:

--- arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-3.dts
+++ arch/arm64/boot/dts/allwinner/sun50i-h6-orangepi-3.dts
@@ -152,13 +132,17 @@ &ohci3 {
&pio {
vcc-pc-supply = <&reg_bldo2>;
vcc-pd-supply = <&reg_cldo1>;
-    vcc-pg-supply = <&reg_vcc_wifi_io>;
+    vcc-pg-supply = <&reg_bldo3>;
};


&r_ir {
status = "okay";
};


+&r_pio {
+    vcc-pm-supply = <&reg_bldo3>;
+};
+
&r_rsb {
status = "okay";

So I was finally successful.

Now I have a fully functional build LE10 for OPi3, including the necessary DT patch. And my experience so far is very positive.

The board is completely stable, no system freezes. It seems to confirm that the official file sun50i-h6-orangepi-3.dts is not correct. In any case, it does not correspond to a publicly known wiring diagram.

Quote from jernej

I have a feeling that this is not so common issue, otherwise there would be more complaints, either here or IRC #linux-sunxi.

Yes, that also amazes me a lot.

Here is the my patch for LE10: 0021-arm64-dts-allwinner-h6-OPi3-stability-LE10.patch