People have to go somewhere. So either you create facilities that everyone can, in principle, use, or you tell a not insignificant section of the population that they aren't really welcome, one way or another, in public society (because they aren't welcome in rather important public facilities).
Which is it to be? Or is there a compromise solution? But then this also makes the mistake of thinking that most people care. And, perhaps a touch ironically, makes the mistake of thinking that as soon as you allow transgender people the privilege of using the toilet, men everywhere will see this as an open invitation to throw on a dress as cover for sexual abuse. Odd, really. I thought it was the liberal left that were supposed to brand all men as, by nature, sexual perverts only a few seconds away from acting on base instincts.