Right-click on a blank area of the desktop and choose Properties. Click the Desktop tab, then click Customise Desktop at the bottom. This'll give you a page that lets you remove the My Documents, My Computer, My Network Places and Internet Explorer icons.
If you want to make further adjustments, Microsoft's free TweakUI may help
http://tinyurl.com/2meyw
If you're trying to remove something else and it won't go, it might be virus/trojan related, and you're advised to follow the usual antivirus / spyware removal procedures.