The simple answer to your question is 'No, he can't'.
The only legislation which bars you from moving to (or travelling to) other places is that which refers to leaving the country. (There are no restrictions as to where you can live within the UK). While not directly relevant to your question, I'll summarise the law for you:
If your partner has parental responsibility and you've not got a residence order, you can't take your daughter out of the country (even for a day) without his permission.
If your partner has parental responsibility and you've got a residence order, you can take your daughter out of the country for up to one month to a time without his permission.
If your partner does not have parental responsibility (because his name does not appear on your daughter's birth certificate and he's not otherwise acquired such responsibility) you don't need his permission to take your daughter anywhere in the world, for as long as you like.
Chris