Yes it's often said it's the one you grew up with, in which case for me it'd be Jon Pertwee, and I do have fond memories of the show with him in. I liked his military friend the Brigadier as well.
I just re-watched the 1st series of the revival and enjoyed Christopher Eccleston's performance more than I remember from seeing it at the time.
I haven't bothered with much of the Smith era but more from getting a bit bored with the show rather than anything to do with him. I'm amused by people dismissing him as "weird" - the Doctor is actually supposed to be a bit weird, what with being a very old alien.
I agree Amy Pond was annoying although I loved the hair, accent and legs. But the latest companion Clara, while similarly easy on the eye, is annoying times a million. Her "pert" routine wore out even before her first episode was over.