I find out that the problem only exists with some video files. One of the "bad" files has this parameter:
A sample (e.g. 2 minutes) of that file, assuming it still shows the issue may be useful.
I can't see anything obvious in the codec options.
Only strange thing is the problematic file was 50fps, and the converted file was 29.754fps.
I don't know if you specifically chose that in encode options, or if that was auto detected (which may suggest an issue with timestamps of frames).