I think you have to decide how it makes you feel. It's all well and good everyone saying 'dont worry about it' but if it does bother you, it won't just magically stop bothering oyu because someone says it should.
So have a think about it; is it a dealbreaker??
He did cheat on you. And that's not very nice. But the relationship is on a very different level now and as you said, you don't think he'd do it now. Whilst that doesn't make it okay that he did, it does mean you have an awful lot more to lose (as well as having a child who anything like that would also affect).
So decide how strongly you feel about it. if you need to try and understand his motivations at the time, then ask him.
I think if I were in your shoes, I'd take it quite hard but in my heart of hearts, know that we'd move on from that.