This is probably best answered by examining possible scenarios:
Let's assume that you have a serious accident when you're driving the car on your own. The accident is your fault. (Everybody has moments of inattention, so it's not impossible!). You and the occupants of the other car are all seriously injured. Your insurance will pay compensation for the injuries to the people in the other car but you won't get anything to help you cope with your injuries. (You'd need separate accident cover for this). So the insurance company only has to pay the people in the other car. (Remember that, for injuries resulting in permanent disability, the payouts could be into 6, or even 7, figures for each person).
Now, the same scenario, except that you've got three young children in the car with you. Once again, everyone is seriously injured. As before, the insurance company will pay out for the injuries to the occupants of the other car but not for your own injuries. But what about the injuries to your children? These will have arisen because of your poor driving and so, all three of your children will then sue you! Yes, I know it sounds a daft idea that a pre-school child will sue his own mother but this is effectively what happens. In practice, of course, there's nearly always an 'out of court settlement' when your insurance company agrees the amount to be paid to each of your children. Once again, this could mean settlements running into 6 or 7 figures.
So, if you often drive with children in the car (as most mothers do) the risks to the insurance company are much higher than if you don't. The result? Higher premiums.
Chris