It would depend very much on the terms of your temporary contract and the nature of the work. There should be something in it about how much notice each side is required to give. Also, whether it's with a temp agency or not, because temp agencies tend to be a little more relaxed over these things.
If you get on well with your current employer, it's nice to give them as much notice as possible. They will remember that consideration when you next ask for a reference.
Ultimately, you could just leave. I don't think they'd be entitled to withhold your wages. It just could cause a little bitterness, though.