Only if there is an easement written into the land certificate of the end house specifically granting a right of way. If not, next door must keep out altogether, otherwise they can be civilly sued for trespass and/or criminally charged with trespass and harassment.
There`s usually something written into the title deeds stating exactly what LIMITED rights of access your neighbour would have [eg access to his garden shed, moving his wheelie bin out to the pavement etcetera].
I would be very surprised if he or his family were to have UNLIMITED access.