Hi Whiffey.
You should change towels regularly, usually once or twice a week. Although you are drying clean skin, dead skin cells will come off on to your towel and 30 drys later thats not a nice thought that all that dead skin has been sitting festering on the towel and each time you have used it you have been rubbing the dead stuff back on to clean skin.
As for taking towels in from the rain, the trouble with leaving any clothes out in the rain and then not washing them again makes them likely to smell a bit stale.