I think there should be a bar so you can change the volume of the casters that's easily accessible in the spectator box.
This should be both for live games and replays.

The only option there is now, is to go into settings and turn down the voice volume from there, which isn't optimal.