I got the impression that Mr Ayrault was losing his bottle when it comes to facing Boris and negotiating with him. He is lucky he won't be facing Nigel. He certainly knows how to tell the Europeans whether to laugh or not!!
I don't think a slander suit is likely. In different circumstances an individual might seek damages, but as someone with a role of creating good international relationships it would be best to treat such insults with the contempt they deserve. The job in hand takes priority over personal slights.
At least Boris speaks 8 languages, including French, whilst the French minister obviously couldn't manage much English. I've been rather hoping he got this job, I think he'll be rather good at it.