That would depend on how well the other people that have used the facilities have washed their hands wouldn't it?
Lets be frank about this - if you need to move your bowels, rather than just pass water you are going to touch the toilet seat (probably), your private parts, the door handles, the handle to flush the loo, the taps, the soap (or whatever container it happens to be stored in if its liquid soap) and you are concerned about the towel? (Oops sorry just noticed your name is violetblue, and realised ladies sit all the time. apologies)
The last thing that is touched before you leave is the door handle out - what about if the person before you hasn't washed their hands and has done something unpleasant ? I try to avoid public loos if I possibly can and I'm a bloke!