I don't particularly object to being called a 'Brit' - short for Britannique - in fact there is a 'Brit Hotel' chain. So I don't see why others should object to their racial descriptions being shortened. I come across casual racism against me every day - les biftecks is the mildest - and yet we say 'frogs'. No problem.
There is a deeper thread, however (very deep in fact) and I am very conscious that it exists. No matter how well I speak French, no matter if I pay my taxes, no matter that I have friends and neighbours with whom we socialise, help and get along - I am still an incomer/immigrant and most importantly NOT French.
I may have drifted off-post here, I started with trying to agree that shortening British to Brit is as bad as Pakistani to ***. Personally I accept Brit, so think that others should accept ***. It's an abbreviation, that's all. If some folk use it badly then that is another thing.