The way I've heard it is that the Swiss and Austrians are disappointed that England are not there, simply because of the interest and revenue generated by each of the "big" teams who do take part.
Whatever anyone may think of England, and however lowly our current position in the world rankings, any European tournament is inherently devalued by England not being involved.
I don't think they were looking for a "nice" tournament with no nasty fans. They were hoping for a major footballing spectacle.