Unfortunately, this happens all too often with Tax Credits. The "rule" they generally work to is that even if they accept that the overpayment is due to their error they still expect the claimant to pay it back unless the claimant could not reasonably have been expected to know they were being overpaid.
You told them of the change and asked them to stop paying you. They will argue that therefore you knew that any payments you received after that were an overpayment.
Was the whole of the �5000 paid after that?
However, you can complain, and ultimately go to the Independent Adjudicator if you wish. Without knowing a lot more detail and looking at the documents (award notices etc) no-one could advise you on whether this is worthwhile. Try your local CAB and ask if you can see someone with special knowledge of tax credit problems - they may be able to take it up for you.
And don't accept the Tax Credit people's figures without a thorough check - they have been known to be wrong!