There are no ffmpeg patches for this yet. Maybe wait until they are ready? Or you want to be proactive and help us with programming?
I would like to help but I am not an expert in this field (ffmpeg, hw video acceleration and so on).
When it comes to drm format modifiers patches i found this: [RESEND,v5,04/23] drm/sun4i: frontend: Pass DRM format info to input format helpers - Patchwork I do not know if it's all and if it's enough.