Let's be clear about this - with very very exceptions, boundary features (walls, fences and hedges) are not jointly owned. There may be a joint responsibility for maintenance but that's not the same thing. (The rare exception is a 'party fence', which is not a fence at all but a masonry structure on a party wall boundary).
Second point - boundaries shown by the Land Registry are known by them as 'General Boundaries' (again with rare exceptions). These show the legal boundary, but you cannot use the plans to scale onto the ground. The physical boundary - namely the agreed line in the ground between neighbours, which they 'agree' marks the position between land ownership is generally the line of any fence erected - because by its existence each side is maintaining up to the fence.
So as others have said, if he has put a fence down, if push came to shove, in law, a court would probably find that you owned up to the fence - which is clearly owned by him.
If you are in the process of selling, I'd do nothing. I'm unclear how or why this has become an issue for the people buying your property.