Ok, Joe and sax,
if you have an ice cube floating in water, a small part of it protrudes above the water level. When it melts, the volume decrease corresponds exactly to that volume of ice that protrudes above water level. Net result - no change in water level. You can try this for yourself - flot some ice in water, mark the level - and then wait for it to melt.
If the Earth's entire floating ice melted, there would be no change in sea level - but, as CT says, there are large volumes of ice locked up in land-based ice masses. If this melts, the huge volumes of melt-water will run off the land into the oceans - hence raising sea level.
To expand your analogy, imagine you have a sink of water with large chunks of ice floaating in it.. You also have some large chunks of ice sitting on the draining board.
As the floating ice melts, it will make no difference at all to the level in the sink - however, as the ice on the draining board melts, the water will run off into the sink and raise the level of the water.