I think this relationship has changed - as it must - when you moved in, and you are seeing your partner's immaturity for the first time.
In an adult relationship, it is perfectly OK for you to decide it is not for you, and yes, he will be hurt, but not as hurt as if you marry him, and then split up.
Ignore his father's last word, your partner is an adult, and you are his partner, not his mother, he is not your responsibility.
Try a break - move out, and see how it goes, but on no account submit to any emotional blackmail. This has to be about how you feel - if you are not a lot more comfortable than this, it's never going to work.
Good luck.