Like you, I'd rather not have the worry of having my money refunded, I want it not to be lost in the first place. The Government that's paying it back will be funded by taxpayers anyway.
I feel the safest bank is The Co-operative Bank, but building societies are safe too I feel. Maybe I'm naive, but I prefer to deal with organisations that are owned by their members, rather than ones that are just out to make money for greedy shareholders. I don't care if I get the best interest rate or not.