hi anndie83, i ahve been in a similar situation. my partner of 5 years has a 6 year old son from his ex. we now have 2 boys together. his son does not listen to a word i say and can be very selfish and spiteful. i believe that although he is not my son when he stays with his he plays by my rules. when i did tell him off i would have his mom on the phone the following day telling me i cannot tell him off.
i started telling my partner that when his son was over on a sunday me and my boys would go out for the day.i did this for a couple of weeks untill he realised that i was serious. because at the time he only seen his son 2 days per month he hated telling him off so let him get away with everything just to play the nice daddy role. this was then effecting our boys as they were getting told off for things that his son was allowed to do.
my advice is sit your partner down and talk to him, tell him how you feel, it took a long time for my partner to see that it was effecting us as a family. i believe that just because you set boundaries it does not make you a bad parent and that the child may respect you more when they are older for giving them boundaries and not leting them do as they please.my partner can now see this and does tell him when he is doing wrong. if you dont speak up you will never have a strong relationship