The most likely answers to your questions are 'no' and 'there are no grounds'. However, please confirm:
you are talking about England & wales
the right (the legal agreement) was registered with the Land Registry and thus appears as an entry on the land title of the land that you hold. There may also be restrictions or obligations on you (as the landowner), for example, an obligation to maintain the track.
Note that these agreements are normally tied to a parcel of land, and not to an individual. They are called easements.