Strange situation. Obviously you should have got this in writing from the landlord before leaving.
Do you still have a deposit with the landlord in one of the Schemes, or has the deposit been returned?
I think it would be very difficult for the landlord to force you to go back now that your tenancy has, in effect, ended. I would say it's up to the landlord to find another tenant and that your tenancy has finished.
It will complicate things though if he still has not released your deposit from a Scheme because he could try to take the rent from for April to June from that.
If you've had your deposit back then I wouldn't worry.