I worked for a (Now defunct) Sporting Goods firm for five years, and never felt like I fitted in. I was tolerated, barely. Felt like the Manager/Assistant Manager were looking for reasons to dismiss me (They never had one), and one of the Supervisors looked at me like I was something she'd just trodden in and ruined her expensive shoes. She seemed fine with everyone else. The Supervisor stayed, but the Management team changed (About three years in) and I never felt that fear again.
Sadly, that job ended in redundancy, so I never got the chance to consider my options, although I feel that if the first Management team had stayed there at the end then I wouldn't have been.