That's true, and is more problematic in a country like the UK which is (a) more densely populated, and (b) maybe more politically-charged. I'd say overall, early on, there was a lot of goodwill and sympathy for the politicians in charge no matter which party you supported, but that is clearly fading -- either because of mixed messages or because of whatever other reason.
The advantage the Swedish had is that the rules were set by an independent body, which provides some level of trust. It would in the UK have been true of any party in power, but if you are motivated even slightly by "what will the voters think?" when making health policy decisions, it has to be the wrong mindset.
A good way of viewing this would be to compare Public Health England with the Swedish equivalent. Over there, the Health Agency is solely and entirely responsible, even to the point where politicians passed legislation that the body required; but that means that they are the ones accountable. This doesn't seem to be the case in the UK. PHE gathers data and implements policy, but does not actually set that policy -- at least, as far as I'm aware. This means that the Health Secretary is still in control of the policy direction etc, but has also meant that PHE is the one held accountable when the policy that it didn't design goes wrong. Hence why PHE has been blamed for various failings, including by the Government.
I'm careful to say "Health Secretary" rather than "Matt Hancock", because I think this is a structural flaw rather than a political one. The theme running throughout this pandemic, that "no Government could have done better/differently", could be just as well expressed as "politicians have nothing particularly useful to add to public health crisis response". If that's the case, why not set up an agency that is responsible not just for implementing health policy but also for setting it?