It's just political point scoring. If it's one of "your own side" saying it, then it's trivial. If it's one of "the other side", then it's terrible.
Any Labour supporter who thinks Ken Clarke's words were terrible should picture how they'd feel if Ed Miliband had said them. Any Tory supporter who thinks there was nothing wrong his words at all should do likewise.
From my own POV, I think his words were unfortunate but another inevitable consequence of our delightful English language. That's to say, the word "serious" can be used in different ways. One of the ways is "extremely bad", as in a "serious accident". Does that mean that there can be trivial accidents? Yes.
Rape is a serious issue, but Clarke wasn't using "serious" in this context. He was using it in the context of the extreme end of the spectrum, like "serious accident". The fact that there are a range of sentences for rape demonstrates that there is such a spectrum, and it's OK if politically clumsy to describe the far end of it as serious.
There was an interview on R4 Today this morning in which a Clarke protagonist (Laurie Penny of the New Statesman) caught herself saying that Clarke showed "serious mismanagement". She seemed to realise that she was using the word "serious" in the exact same context as Clarke - but I think she got away with it ...