This happened in America a while ago, and the judge found for the person claiming- but that was because the lady had been talking to the person since the split and informally agreed (either via email or text, I forget which) that she would pay him back. It was along the lines of 'yeah, I know owe you' or some such "proof".
If he doesn't have any proof of this kind at all, this will be much harder to prove. I wouldn't say you're 100% safe on this (although I know that's what you need to hear), but my advice is just to ride it out. Without evidence, it is unlikely on the 'balance of probabilities' he can prove you had the 'intention to create a legal relationship'- and he would need to show this in order to prove you two had a contract, however informal, between you.