I'm fairly sure I pay tax on the small amounts of interest I earn in bank accounts. It's small enough that I could claim it back as part of the allowance. I've never got around to sorting that out. Just lazy, really. I expect many people are closer to lazy and/ or not really thinking about tax-dodging, rather than doing it deliberately or not at all.
Don't even think it's about the "punitive" tax system. Given a choice between paying £100 cash or £110 by card, many people would still go for the cash option, and a 10% VAT is hardly punitive. People just want to save money.