Most modern PCs have two On/Of switches: One at the front, and one at the back by the mains input. The one at the front is not actually an on/off switch in the traditional sense. It is a monentary switch (i.e. it only makes a circuit when you are actually pressing it.). This is connected to some circuitry on the motherboard. When you press it it sends a signal to shut down the computer. If the machine is off, it sends a signal to turn it on. Obviously to send a signal, there must already be some power there to monitor the circuit.
When you turn off using this switch, although you are not putting the machine into Standby in the computing sense, you are (believe it or not) putting it into standby in the appliance sense. That is to say, it is rather like turning off your TV using the remote control. The only power being consumed is that used by the monitoring circuit.
If you were to take the cover off the machine, you would see that there is a single LED glowing on the motherboard.
So, the only way to fully turn off the machine is with the switch at the back (after powering down using the front switch or "Shutdown").
Some cheaper power supplies don't have that switch at the back, in which case you must pull the plug!