I've yet to use it on anyone, but a great insult is "I would challenge you to a battle of wits, but you've come only half prepared."
The worst thing I've ever been called is specky, when I used to wear glasses. It's not much, but I hated having to wear them, so that didn't help. I've since had my eyes lasered. My best pal was once heckled by a bunch of Neds (Glaswegian chavs) who shouted at him "Ho specky, whit's wi the glesses, can yeh no see?" He replied, "Well, no." I guess that would have been a perfect time to have used my above perfect insult phrase, but he probably would have got a doin' (in the parlance of our times).