Comets aren't just covered with ice, they are made from it. When the solar system was formed from its planetary accretion disk, the centre of gravity was formed where the sun is. The particles and atoms in the disk sorted themselves by density, the dense ones falling in towards the centre, the lighter ones staying further out. This is why planets toward the centre of the solar system are dense and rocky, those further out gas giants. Furthest out of all, the oort cloud, is mainly water. This is where comets come from.