Cache settings can smooth out the ebbs/flows of a wireless connection where you get sudden drops in bandwidth, because buffering more data helps paper over the gap when it happens (and then normal throughput resumes). However it can *never* solve the more fundamental problem of there not being enough bandwidth for a sustained large file transfer. In fact, since you force it to queue more data before play start, if you saturate the link and the supply of data cannot keep up, you will also wait longer for it to resume because we need to buffer into a larger cache. Users always add cache and this negatively impacts the problem, ergo reducing the cache size might improve it, albeit there's a razor thin margin involved. All connections fluctuate a little, so a little cache always helps, and by default Kodi uses a little cache.
TL/DR; If the network can't keep up the solution is always to fix the network not fiddle with cache settings.