Probably conflicting advice really.
Firstly, on a personal level, if I am happy with something, I have learned the hard way, don't change it, no matter how green the grass seems.
I don't think it will affect you credit rating, as to some extent this is shared info. You do, as you say, have a long relationship with them, which I agree SHOULD count, but that is not always true.
I was actually with the Halifax for 20 years. Still am on paper, because a year after I asked them to, they still haven't closed my account and refunded me the money in it. I asked them to close it because they had the cheek to impose bank charges when they withdrew my overdraft without telling me. They say they did, but they did so by a web letter, and for three months I had no PC for anything, so I was not aware. Whatever, the long - and mostly good - relationship amounted to nothing. As such I have finished with the halifax for life.