Your male cat is definately scent marking & yes it's because just as each gets settled in their area you swap them over. He is still trying to intimidate your girly through scenting everywhere!
I have two suggestions. First allocate each cat a 'core territory' for example a bedroom each. Only allow access to that area to the 'owner'. Never allow the other cat in. Then they should be able to 'share' the rest of the house. Set up a few boltholes for both cats - a small cardboard box with only 1 cat sized entry/exit, concealed/placed behind things like the sofa or on top of a wardrobe. Increase the 'resources' for both cats. More water bowls, feeding places, resting places & viewing places, making sure they are spread around the house. This eliminates a lot of the conflict of competing for these things.
Then the second suggestion - using a pheramone spray to change the marking behaviours. This can be purchased from your vet or online, the best known brand is 'Feliway'. It comes with full instructions to change clawing & urine marking to facial rubbing. Hope this helps.