Loopholes generally apply to very old covenants where the person(s) for whose benefit the covenant was imposed cannot be traced. This is clearly not the case with you.
The precise wording of the covenant can have a bearing on whether it is enforceable. You would need advice on this from an experienced property solicitor.
The alternative is to agree a deal with the previous owner to split the profit from development, if planning approval can be obtained. After all, he is not going to get anything unless you agree to the land being developed - he can't make you do this, so he has something to gain by compromising. You would need a good land agent to value the land with permission for development.