1. Lateral thinking. Cut the stems off above the polymer and fill with decorative glass beads or stones.
2. As a chemist, if the vases are pottery or porcelain you might try heating them to melt the polymer. Don't do this indoors in case this releases fumes. Know anyone with a bonfire? Wrap in foil before putting in hot embers. You could try and chip a little off to see whether melting would work.
3. Dissolving the polymer in a solvent is messy and not to be recommended.
I like solution number one myself.