AOG
I don't see any problem with this. It's something that we all do either consciously or subconsciously. I will always book business flights when I'm travelling so that I don't have to sit any where near young children (noisy) and other certain demographics.
I choose to live in a nice part of London and I choose to associate with nice liberal-minded folk, rather than boorish knuckle-draggers.
There's nothing racist about what these people are doing. I'm sure if the CEO of a private bank, who happened to be black or Hispanic, were to buy a house there, they would receive no opposition.
Similarly, if a white trashy chav (or the American equivalent) were to try and 'get through the gates), they would meet with some resistance.
AOG - I am sure that you would be happy to have (say) Sir Trevor McDonald living next to you, than White Dee from Benefits Street, right?
It's not racism...it's common sense.