For complete albums the Album art is held as hidden files in the folder.
To delete it you need to display hidden files, then delete the files with names like AlbumArt, AlbumArtSmall, Folder.jpg and desktop.ini.
You can select the album art you want as follows:
1) Open Windows Media Player
2) Find the Album you want to change or add the artwork for
3) RIGHT click and select Find Album Info
4) Press the Search button and search for the album / artist
5) Once you find the album/tracks you want press Next
6) This will display your final selection (sleeve / tracks) press Finish
An album can have multiple versions in different countries, some with extra tracks or different trakcs, so it can take a while to find the right album.