Ive often thought about asking the same question, I wear a T shirt every day of the year, if it is bitter cold I put on a light jacket, I never wear gloves, I do have a thick heavier jacket but that,s just to keep the driving rain out.
People look at me a bit strange when they have loads of layers on and I am wearing a T shirt in the winter.
I admit that sometimes I feel a little cold when out but it is a bearable cold, I think the problem is, people aren't willing to tolerate a bearable cold, they want to be warm and snugly in the depths of winter, that's fine but it does not build up any resistance to the cold.
I remember leaving work at 22.00 hrs in the winter and started walking home in my T shirt with my jacket over my shoulder, I was really enjoying the brisk cold when Carakeel pulled up in the car telling me I was mad as the temperature was -6. I was waking briskly and feeling a little chilly, I was fine.