A lot of computers keep the USB ports powered up when the computer is off these days, it's useful for charging things like phones/tablets
To stop it you can firstly try plugging the hard drive into a different USB port, as often it's only one or two that provide power when the computer is off (often the front ones, so if it's a desktop computer try plugging the drive into a rear USB port)
If that doesn't work there is normally an option in the computers BIOS to enable/disable power to the ports when off, but without any details of what the make and model of computer is though it's not possible to give any useful advice on how to change this.