Yep - though I guess you're then in a world of Rec 601 vs Rec 709 vs Rec 2020 YCrCb conversions (as they are all different) if you play back a Rec 601 DVD and output it in Rec 709? (And that's ignoring gamut conversions - which you kind of can with 601/709 tolerably, but you can't with 709 vs 2020)
Arguably taking stuff back to RGB rather than doing YCrCb.601 - > YCrCb.709 conversion in the YCrCb domain is a six and two threes?
My understanding is that for HDR content there are flags in the HDMI stream to indicate the color space. I think that takes care of 709 and 2020 for folks with a HDR capable display. The source doesn't have to do any conversions, just enable the right flags in the output. I'm not sure how 601 is handled.
There's going to need to be a lot of LE work done if people are expecting the RPi 4 to play pretty much everything and get reasonably correct results on all display types. That will require handling just about about every possible color space conversion, HDR to SDR tone mapping, and who knows what else.