I had the same issue on a pi3. The issue was that when running both tvh client and server on the same pi AND using advanced deinterlacing method - green artefacts would appear from time to time, also continuity counter errors in tvh log.
The solution was to use BOB deinterlacing instead of "advanced". However, BOB looks much worse than advanced.
Another solution was to increase the arm_uc value, e.g.:
I don't have pi4 so no idea if this would work.