DVB issue since LE switched to kernel 4.9.x

  • any progress? just tested latest kernel 4.13.12 on x86 +S950 and it's still affected

    Looks like it's I/O dependent: the problem manifests after tens of minutes when just displaying DVB stream (cca 90mins) but with heavy disc I/O in serveral minutes...

    EDIT: reported upstream: 197835 – DVB-S discontinuity, PES packet size mismatch, stream corruption

    Confirmed, any progress here?

    I could test builds on rpi3.

    My dvb card is technotrend s2-4600.

  • Over the past couple of months I've been bisecting and debugging the kernel on my Acer Revo 3700 Generic x86_64 machine. This was quite a learning curve, as I haven't previously used git very much let alone bisect a kernel. However, I found there were plenty of guides around on the internet, so I was quickly able to get upto speed. I must also say that the build system for LibreELEC is extremely well put together and easy to follow.

    After approx 50 builds I believe I've found the answer....I've hit a few issues along the way - the main problem being that at several commit points in the 4.9 rc1 development the kernel would not compile due to breakages in the netfilter module, and this was preventing me from getting close to the troublesome commit. I therefore disabled netfilter in the kernel options, and from then it only took a couple of weeks to locate the troublesome commit.

    Here is my git bisect log:

    So this indicates the troublesome commit is: 4cd13c21b207e80ddb1144c576500098f2d5f882

    To test this theory, I've created some builds based on 8.2.2 with a kernel patch applied to revert the change made under 4cd13c21b207e80ddb1144c576500098f2d5f882.

    LibreELEC-Generic.x86_64-8.2.2-dvbfriendly-nomb.tar

    LibreELEC-Generic.x86_64-8.2.2-dvbfriendly-withmb.tar

    LibreELEC-RPi2.arm-8.2.2-dvbfriendly-nomb.tar

    LibreELEC-RPi2.arm-8.2.2-dvbfriendly-withmb.tar

    I've tested the Generic builds on my Revo 3700 which appear to work... no artefacts after running for a couple of days, whereas I'd normally see artefacts every 2-3 minutes with any of the 4.9+ kernel-based builds :)

    The Generic build also contains a further kernel patch to address the buffer overflow problem in timer.c as discussed on page 8 of this thread (Kernel 4.9.x drops USB data on Pi 2B (regression from 4.4.x) · Issue #2134 · raspberrypi/linux · GitHub). Since I found this also has impact but to a much lesser degree - without this you may get artefacts maybe 2-3 times per day depending on usage. The v4.9.59 kernel shipped with 8.2.2 for RPi2/3 did not require this patch as it's already fixed in that version.


    Maybe others could perform some testing on these builds (especially on the Pi2/3) to confirm whether this does indeed resolve the issue.

    In the longer term I have no idea on the best way to fix this - it looks like the commit was made to quite intentionally send all I/O traffic onto the ksoftirqd process queue rather than processing the request immediately. This seems to have consequences for lower-spec devices where you are trying to use DVB cards. My Revo 3700 machine has a quad core 1.8GHz processor and I have a fairly minimal LE build with only a handful of addons installed. It works perfectly with 4.8-based kernels, but suffers badly with artefacts on unpatched 4.9+ kernels, with picture breakups every 2-3 minutes, or even more often when put under heavier I/O load. This seems to fit with what others have reported in this thread - i.e. the problem surfaces when under heavier I/O load.

  • jahutchi could you test if the issue also occurs on your Revo with other Linux distributions running a current kernel like Ubuntu? And if reverting the same commit fixes it there?

    Next step then would be to check if the issue also occurs if the DVB device is accessed directly (eg using VLC, Kaffeine or some other DVB player) or if tvheadend plays some part in it.

    Then that needs to be discussed with upstream (dvb driver / tvheadend / kernel maintainers).

    so long,

    Hias

  • I tested on RPi3 and I can confirm that reverting commit 4cd13c21b207e80ddb1144c576500098f2d5f882 completely fixed the issue.

    ^ to be pedantic, it works around the issue, it does not fix the issue. Adding patches is only a short term solution and we don't want to add patches all the time as it impacts long-term sustaining of the distro. At least we now understand where the issue lies.

  • @jahutchi thank you!


    While we would still like a real solution (ideally upstream kernel devs acknowledging the issue introduced and finding a solution), identifying the commit that caused the regression makes this far more likely.

    And it allows people affected to run a build much closer to the official one - running the 4.8 kernel was never going to be a solution that would work indefinitely.

    Good work!

  • jahutchi  smp

    I reported the regression upstream Aw: Re: dvb usb issues since kernel 4.9 — Linux USB

    Could you please follow/join the conversation and maybe provide further information ([email protected])?

    Alan Stern had the following questions:

    It would be good to know what hardware was involved on the x86 system
    and to have some timing data. Can we see the output from lsusb and
    usbmon, running on a vanilla kernel that gets plenty of video glitches?

    Overall, this may be a very difficult problem to solve. The
    4cd13c21b207 commit was intended to improve throughput at the cost of
    increased latency. But then what do you do when the latency becomes
    too high for the video subsystem to handle?

  • This is awesome! Big thanks to everyone involved. I was finally able to update my Raspberry Pi 3 LibreElec from 7.9. I am using several USB DVB-C tuners (model: Astrometa Panasonic MN88473 dvb-c). Those official 8.x LibreElec releases all caused same kind of stream issues as described in this thread.

    But now I upgraded to image provided by milhouse (this one: LibreELEC-RPi2.arm-9.0-Milhouse-20180102133316-#0101z-g866291e.tar ). Tested now half a day and looking really good: no errors yet. Hope this will be fixed in upstream also some day.

  • But now I upgraded to image provided by milhouse (this one: LibreELEC-RPi2.arm-9.0-Milhouse-20180102133316-#0101z-g866291e.tar

    You can now use Milhouse' nightly builds - the ksoftirqd patch is included since build #0104.

    Edited 2 times, last by smp (January 8, 2018 at 11:12 PM).

  • Presumably the VDR issue is unrelated to the kernel patch.

    This must be somehow related, I never had an issue with VDR before. No clues in kodi.log or "dmesg | grep dvb". The tuner seem to be enabled but VDR backend just doesn't start.

    I will try it with my own build.

    Edited 4 times, last by smp (January 8, 2018 at 5:38 PM).