Thjis guy has been keeping his options open - men are good at that because they can enjoy a sexual relationship with far less of a depth of commitment than a lot of women.
So, having been discovered, it's time to cut your losses and walk away.
Of course that is very very hard, and for quite a while you will be tempted to reconcile because a faulty relationship feels better than no relationship at all.
The truth is, that is in illusion, and will simply store up more pain for a future time.
A quick clean break now will hurt a lot, but is still the right thing to do - you deserve better than to be kept as a 'maybe' in case is other relationship did not work out - and that shows what kind of man he is.
So break, and stick to it. Don't mail him, don't speak to him except within the bounds of courtesy, and don't respond to any contact at all. It's the only way to get past this sad time in your life, so look forward, and find a man worthy of being loved by you - this guy is not him.
As for contact with the other girl - a natural reaction, but she may well be innocent and unknowing of your relationship. In the interest of dignity, avoid contact with her - a complication you don't need right now.