and free cooling courtesy of the icy death vacuum of space, is it really that bad?

Yes. You don’t get cooling AND vacuum from space, the vacuum aspect prevents the majority of cooling since there’s nothing to take that heat away other than IR radiation.