Contrary to popular belief, not wearing prescribed glasses will not adversly effect your eyes. All that will happen is that you will get headaches, wrinkles from squinting (as has already been said).
Glasses are NOT a cure, if you think of your eye as a telescope that will not focus, glasses provide an extra lense so that it can focus properly and that is all they do.
Having said all this, why would you not want to be able to see more clearly? Are contact lenses an option for you?