I use diluted vinegar if they around the body of the glass. Forget about it for a day or two. If they are where a small amount has been left in the base I put rice in the bottom and swirl it to massage off the watermark ( good in decanters too! )
Soaking in diluted vinegar will help, as will soaking with a Steradent denture tablet overnight. However, I've found that if you live in a hard water area, after a time the lime and other chemicals in the water do tend to leave a permanent stain which is difficult to remove and always make the empty vase look unsightly. For that reason I've now given up using crystal vases and use pottery/ceramic ones which hide the ugly water marks on the inside.