I think it's about the impression you make as you walk in - and that is what is very important for a candidate.
If I were interviewing, I would expect the interviewee to be smartly dressed - certainly not in jeans and a tee-shirt.
To dress smartly shows a degree of respect for the interviewer, and the situation, and in far more cases, you are likely to make a negative impression if you appear casual. If the interviewer doesnt mind either way, you have lost nothing be being casual, if they think as i do - and more of them do trust me, then you are off to a bad start with your self-image before you have opened your mouth.
Discrimination is a whole other area, and does not apply in this case. You may not be selected because you gave the apearence of a lack of intent to impres by being casual - thaty's not discrimination, it's a valid assessment of your potentlal worth as an employee.
Remember, it's their bat, their ball, their game, their rules.
You may stand proud for having been true to your ideals, but you can still be unemployed because of it, so that is a Pyrrhic victory.