xkcd

Water from non-Earth sources might contain dissolved minerals at poisonous levels for agriculture, much less human consumption.

Oh yeah, it's practically guaranteed to contain nasty stuff! We're gonna drink it anyway though.

Most of that water on earth that we'd consider "not useful" would fall into the "100% useful" category if found in space. As long as the contaminants have a different boiling temperature from water, you can always boil the water into steam in order to separate it. Or you could also use electrolysis to separate out the hydrogen and oxygen and then recombine then in clean tanks.

These are expensive methods of purification, energy intensive, but solar panels really well with no atmosphere and 24/7 sun exposure, so this is all feasible.