In my experience people seem to generally be in favour of Afghanistan. It's Iraq people are (rightly) furious about.
However, I don't think withdrawing is the solution at all. It's not only utterly spineless but also incredibly destructive. Fortunately the US has a decent commander on the ground in Petraeus.
Blair would have likely been remembered okay had it not been for Iraq. And he could've even come out of that better than he did by not lying to the UK's sovereign representative body...
In all honesty, I think it'd be hard for any leader to retain popularity after a decade in office (exempting some of course) - and if the party understands that popularity is slumping, they'll pressure them to leave...