Unless stated in the deeds that a fence MUST be erected and maintained then no one.
Other than the above the person with animals has a duty to keep their animals fenced in.
You can download a copy of yours and your neighbours deeds from the land registry website for about £5 eash. This should indicate with a T whos responsibility it is and if there are any conditions set.