Or right click on the taskbar and choose "Properties". At the bottom of the window that then appears, tick "Hide inactive icons" then to the right of this, "Customize".
You'll then see a list of programs which are currently in your taskbar and you can choose to "Always Hide" them, or "hide when inactive" etc.
All of this may hinge on your operating system : I have XP Pro.
Also, what I've described leaves the program running but hides it from the taskbar rather than closing it.
If you want to stop a program launching at startup and putting itself onto the taskbar in the first place, follow the earlier advice from peanut.