You are not alone in this situation. My brother also never married and stayed at home with our parents. He's in his fifties too. When my mother died my dad signed the house over to him so the state wouldn't take it off him if he had to go into care (he's 87 and still ok at the moment). My sister and I have our own houses and we are okay about our parents house going to our brother BUT, what if he gets married after dad dies?? His wife will inherit our parents home. My brother has made a Will but we don't know what he's going to do with the house. In any event, if he marries the Will will be null and void. Quite honestly, it's none of my business really. What will be will be.
I would say in your situation you probably wouldn't really want to see him homeless but he should be made to pay the going rate for renting the house surely. Then, if that fails, take legal advice. I would still advise you to keep things reasonable in trying to get the best outcome for all.