It's not the word, it's how it's used and who you use it to IMO.... but it is a word that you have to be very careful about.
I'd happily call my friends tw@ts, I wouldn't call a stranger that.
In my circle of friends I have a friend who calls another friend "Mr. Black" but they've been mates for over 30 years and it's taken as it's meant, I don't know the "Mr. Black" well enough to call him that, so although he's a friend I don't feel I could call him that.