First, let's get it clear what you're now charged with:
Swearing at someone, per se, is not an offence.
Using "threatening, abusive or insulting words or behaviour" becomes an offence under Section 5 of the Public Order Act 1986 when it's done "with intent to cause a person harassment, alarm or distress"
However you've been charged under Section 4, which means that you're accused of using "threatening, abusive or insulting words or behaviour . . .with intent to cause that person to believe that immediate unlawful violence will be used against him or another by any person, or to provoke the immediate use of unlawful violence by that person or another, or whereby that person is likely to believe that such violence will be used or it is likely that such violence will be provoked".
So you've effectively been charged with 'threatening behaviour' or (for example) of using words which were intended to get the other guy to throw a punch at you.
What may, or may not, have gone on before that is irrelevant to the charge (although it could be relevant to any mitigation for sentencing). For (an extreme) example, if you'd just seen a guy rape your girlfriend and murder your children, and that guy was now coming aggressively towards you, if you shouted "Come on then, you stupid fu**er, do you want some?" you would be guilty of an offence under Section 4.
So you need to consider what you actually said. If (even if it didn't use a single swear word) it was said in order to make the guy think that you were going to assault him (or to goad him to attempt to assault you) you have no defence. You are guilty.
Otherwise, plead 'not guilty' and leave it to the CPS to try to prove the opposite.
Chris