Overclocking is the only thing that may help.
VC-1 is poorly optimised in ffmpeg (about 40% of time is spent in C code loop filter functions).
We have had a low-priority piece of work for our codec guy to write some Neon code for this which should resolve this, but the move to v4l2 and drm has kept the codec guy too busy to get to this.
I have got the boss to okay another guy to work on this - so hopefully there will be progress soon.