any progress? just tested latest kernel 4.13.12 on x86 +S950 and it's still affected
Looks like it's I/O dependent: the problem manifests after tens of minutes when just displaying DVB stream (cca 90mins) but with heavy disc I/O in serveral minutes...
EDIT: reported upstream: 197835 – DVB-S discontinuity, PES packet size mismatch, stream corruption
Confirmed, any progress here?
I could test builds on rpi3.
My dvb card is technotrend s2-4600.
Over the past couple of months I've been bisecting and debugging the kernel on my Acer Revo 3700 Generic x86_64 machine. This was quite a learning curve, as I haven't previously used git very much let alone bisect a kernel. However, I found there were plenty of guides around on the internet, so I was quickly able to get upto speed. I must also say that the build system for LibreELEC is extremely well put together and easy to follow.
After approx 50 builds I believe I've found the answer....I've hit a few issues along the way - the main problem being that at several commit points in the 4.9 rc1 development the kernel would not compile due to breakages in the netfilter module, and this was preventing me from getting close to the troublesome commit. I therefore disabled netfilter in the kernel options, and from then it only took a couple of weeks to locate the troublesome commit.
Here is my git bisect log:
git bisect start
# good: [2937f375751922ffce9ef1d5fa84491840b0c8e0] staging/lustre: Disable InfiniBand support
git bisect good 2937f375751922ffce9ef1d5fa84491840b0c8e0
# bad: [6b5e09a748ad0a0b198d0e268c7e689044bfe48a] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
git bisect bad 6b5e09a748ad0a0b198d0e268c7e689044bfe48a
# bad: [5691f0e9a3e7855832d5fd094801bf600347c2d0] Merge tag 'sound-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
git bisect bad 5691f0e9a3e7855832d5fd094801bf600347c2d0
# good: [7667d445fa9e84be028d2f658c537f4f5584d250] Merge tag 'rxrpc-rewrite-20160930' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs
git bisect good 7667d445fa9e84be028d2f658c537f4f5584d250
# bad: [21f54ddae449f4bdd9f1498124901d67202243d9] Using BUG_ON() as an assert() is _never_ acceptable
git bisect bad 21f54ddae449f4bdd9f1498124901d67202243d9
# good: [5419e783829127dba712be769bce8c6a1ec0057e] Merge tag 'm68k-for-v4.9-tag1' of git://git.kernel.org/pub/scm/linux/kernel/git/geert/linux-m68k
git bisect good 5419e783829127dba712be769bce8c6a1ec0057e
# bad: [e6dce825fba05f447bd22c865e27233182ab3d79] Merge tag 'tty-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
git bisect bad e6dce825fba05f447bd22c865e27233182ab3d79
# good: [c9fef1cc3dd3677633e6fd6ea5bd7ef3b741fab3] drivers/misc/hpilo: Changes to support new security states in iLO5 FW
git bisect good c9fef1cc3dd3677633e6fd6ea5bd7ef3b741fab3
# bad: [597f03f9d133e9837d00965016170271d4f87dcf] Merge branch 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 597f03f9d133e9837d00965016170271d4f87dcf
# bad: [4cd13c21b207e80ddb1144c576500098f2d5f882] softirq: Let ksoftirqd do its job
git bisect bad 4cd13c21b207e80ddb1144c576500098f2d5f882
# good: [16217dc79dbc599b110dda26d0421df47904bba4] Merge tag 'irqchip-4.9-1' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into irq/core
git bisect good 16217dc79dbc599b110dda26d0421df47904bba4
# good: [f61f86068cdf7e59de64d430fd3cc907a8e102f2] Merge branch 'irqchip/mvebu64' into irqchip/core
git bisect good f61f86068cdf7e59de64d430fd3cc907a8e102f2
# good: [464b5847e61085f81bb99ce48eb427a0dc7617dc] Merge branch 'irq/urgent' into irq/core
git bisect good 464b5847e61085f81bb99ce48eb427a0dc7617dc
# good: [47f91519546ce39cceee2c51b0f5045eadc688a9] ARM/STM32: Select external interrupts controller
git bisect good 47f91519546ce39cceee2c51b0f5045eadc688a9
# good: [474aa3dd3e1f3ae410115fe6624ba48fc9791bc5] Merge tag 'irqchip-core-4.9' of git://git.infradead.org/users/jcooper/linux into irq/core
git bisect good 474aa3dd3e1f3ae410115fe6624ba48fc9791bc5
# good: [b8129a1f6aaaca02d92186acf19ceb545b4b489a] genirq: Make function __irq_do_set_handler() static
git bisect good b8129a1f6aaaca02d92186acf19ceb545b4b489a
# first bad commit: [4cd13c21b207e80ddb1144c576500098f2d5f882] softirq: Let ksoftirqd do its jobSo this indicates the troublesome commit is: 4cd13c21b207e80ddb1144c576500098f2d5f882
To test this theory, I've created some builds based on 8.2.2 with a kernel patch applied to revert the change made under 4cd13c21b207e80ddb1144c576500098f2d5f882.
LibreELEC-Generic.x86_64-8.2.2-dvbfriendly-nomb.tar
LibreELEC-Generic.x86_64-8.2.2-dvbfriendly-withmb.tar
LibreELEC-RPi2.arm-8.2.2-dvbfriendly-nomb.tar
LibreELEC-RPi2.arm-8.2.2-dvbfriendly-withmb.tar
I've tested the Generic builds on my Revo 3700 which appear to work... no artefacts after running for a couple of days, whereas I'd normally see artefacts every 2-3 minutes with any of the 4.9+ kernel-based builds 
The Generic build also contains a further kernel patch to address the buffer overflow problem in timer.c as discussed on page 8 of this thread (Kernel 4.9.x drops USB data on Pi 2B (regression from 4.4.x) · Issue #2134 · raspberrypi/linux · GitHub). Since I found this also has impact but to a much lesser degree - without this you may get artefacts maybe 2-3 times per day depending on usage. The v4.9.59 kernel shipped with 8.2.2 for RPi2/3 did not require this patch as it's already fixed in that version.
Maybe others could perform some testing on these builds (especially on the Pi2/3) to confirm whether this does indeed resolve the issue.
In the longer term I have no idea on the best way to fix this - it looks like the commit was made to quite intentionally send all I/O traffic onto the ksoftirqd process queue rather than processing the request immediately. This seems to have consequences for lower-spec devices where you are trying to use DVB cards. My Revo 3700 machine has a quad core 1.8GHz processor and I have a fairly minimal LE build with only a handful of addons installed. It works perfectly with 4.8-based kernels, but suffers badly with artefacts on unpatched 4.9+ kernels, with picture breakups every 2-3 minutes, or even more often when put under heavier I/O load. This seems to fit with what others have reported in this thread - i.e. the problem surfaces when under heavier I/O load.
jahutchi could you test if the issue also occurs on your Revo with other Linux distributions running a current kernel like Ubuntu? And if reverting the same commit fixes it there?
Next step then would be to check if the issue also occurs if the DVB device is accessed directly (eg using VLC, Kaffeine or some other DVB player) or if tvheadend plays some part in it.
Then that needs to be discussed with upstream (dvb driver / tvheadend / kernel maintainers).
so long,
Hias
yah.......finally i m so happy me too, i used the last build with an rpi1 and the erros are gone. i had befor around 1 error per mbyte.
thank u so much.....
I tested on RPi3 and I can confirm that reverting commit 4cd13c21b207e80ddb1144c576500098f2d5f882 completely fixed the issue.
^ to be pedantic, it works around the issue, it does not fix the issue. Adding patches is only a short term solution and we don't want to add patches all the time as it impacts long-term sustaining of the distro. At least we now understand where the issue lies.
@jahutchi thank you!
While we would still like a real solution (ideally upstream kernel devs acknowledging the issue introduced and finding a solution), identifying the commit that caused the regression makes this far more likely.
And it allows people affected to run a build much closer to the official one - running the 4.8 kernel was never going to be a solution that would work indefinitely.
Good work!
I reported the regression upstream Aw: Re: dvb usb issues since kernel 4.9 — Linux USB
Could you please follow/join the conversation and maybe provide further information ([email protected])?
Alan Stern had the following questions:
It would be good to know what hardware was involved on the x86 system
and to have some timing data. Can we see the output from lsusb and
usbmon, running on a vanilla kernel that gets plenty of video glitches?
Overall, this may be a very difficult problem to solve. The
4cd13c21b207 commit was intended to improve throughput at the cost of
increased latency. But then what do you do when the latency becomes
too high for the video subsystem to handle?
This is awesome! Big thanks to everyone involved. I was finally able to update my Raspberry Pi 3 LibreElec from 7.9. I am using several USB DVB-C tuners (model: Astrometa Panasonic MN88473 dvb-c). Those official 8.x LibreElec releases all caused same kind of stream issues as described in this thread.
But now I upgraded to image provided by milhouse (this one: LibreELEC-RPi2.arm-9.0-Milhouse-20180102133316-#0101z-g866291e.tar ). Tested now half a day and looking really good: no errors yet. Hope this will be fixed in upstream also some day.
But now I upgraded to image provided by milhouse (this one: LibreELEC-RPi2.arm-9.0-Milhouse-20180102133316-#0101z-g866291e.tar
You can now use Milhouse' nightly builds - the ksoftirqd patch is included since build #0104.
I've uploaded a new build, #0107z, which is based on 4.14.10, no commits reverted, but includes the increased buffer patch suggested in this email.
Please test and report results - thanks!
Please test and report results - thanks!
VDR server does not start with that build.
VDR server does not start with that build.
Any clues in the log? I'm doing this blind - I don't run VDR. Presumably the VDR issue is unrelated to the kernel patch.
Presumably the VDR issue is unrelated to the kernel patch.
This must be somehow related, I never had an issue with VDR before. No clues in kodi.log or "dmesg | grep dvb". The tuner seem to be enabled but VDR backend just doesn't start.
I will try it with my own build.
