It's a fairly common problem. Windows thinks that a program (typically Windows Explorer) is still trying to access the drive, even though it isn't.
As has been suggested, simply unplugging the drive with your computer turned off is all that's required. However there's another way of dealing with the problem:
Install this little utility on your computer. (It's on mine):
http://www.softpedia.com/get/System/System-Miscellaneous/Unlocker.shtml
When a drive is incorrectly shown as being in use, go to Windows Explorer, right-click on its icon and select 'Unlock All'. You'll then find that you can eject the drive in the usual way.