I have a couple of cards that I only ever use online - I don't even know the PIN - so the actual physical card is something of a liability (say if it gets lost/stolen).
I just photocopy the front of the card & keep a separate record of the CV2 number (from the signature strip) - much safer.
I did the exact same thing earlier this year when my bank sent me a new card. When I phoned the bank and rather sheepishly told them what I had done, the girl I spoke didn't seem at all surprised and said that it happens all the time !
I told the bank I'd never recieved the nice new debit card they'd sent out...so they cancelled it and sent a new one.
I found the first one about two months later...bundled up with other expired cards. Ooops
I thought I was the only one to have done this. I had to phone the company and grovel an apology and I was told it was ok, it happened more than you think.