I had a similar experience with the Co-op Bank years ago.
They messed me around something rotten, so I decided to move accounts. I arranged to transfer my loan account to my new bank, but the Co-op cocked up, and put �2,000 in my current account. I withdrew it in cash, and when they asked for it back, I told them that I was going to be similarly inefficient - specifically, every time I had to ring them, I would deduct �5 in charges from the balance, and every time I wrote to them, I would charge �20 charges - similar to what they were doing to me.
A flurry of corespondence was exchanged, culminating in them sending me a very terse letter demanding the money back, and charging me �60 for that one letter.
I obtained legal advice.
The upshot was, my solicitor told the Co-op that i had withdrawn the money in good faith, and was willing to repay at �30.00 per month. All charges were to be withdrawn, and all correspondence from them to me was to cease - failure to comply would result in involvement of the Ombudsman, and their faulty practice would be exposed.
They gave in - because I offered reasonable repayment terms and the error was theirs, they had no legal recourse.
So - if you fancy keeping the money and offering to pay it back at say �30 - �40 per month, I'd give it a punt.
Up to you.