Just on a point of order, interpreters work in courts as you describe. An interpreter translates orally, while a translator interprets written text.
Yes all court users have to adjust their method of address when an interpreter is involved. Often overlooked when the defendant has an interpreter is the fact that the defendant has to be able to understand everything that is said and not just that addressed to him. Other court users often forget this when, say, questioning a witness (who is not in need of an interpreter) and plough on, leaving the interpreter struggling.