You need to tread carefully here.
Even though your partner and his wife are not together, and haven't been for some time, they are always going to be in touch (assuming she pulls through) because they share the children.
Having children with someone creates a powerful bond which is usually not severed by divorce, for practical, and as you are discovering, emotional reasons.
I think you need to make allowances for your partner's feelings for his ex-wifef, which may be simply rooted in caring for someone he used to love, even if he no longer loves her in that way.
Remember, he is not with her, he is with you, and in these early stages, it needs everyone to take it slowly and adapt to the changes, including the children, who now see someone else with Dad, and need to assimilate that change for themselves.
So avoid a 'sit down' and go with the flow where possible.
Make sure you do not allow anyone to see you as a 'substitute' mum, including him and the children - you are your own person at this stage, and everyone is going to take time to see what fits, and what doesn't.
Take things a day at a time, and remember, your partner's first concern is going to be for his children, and that will extend to a degree to his ex because of her relationship with him, and them.
It's complicated, but follow your instincts. Try not to look to draw 'ownership' lines with anyone, and you will find that if this relationship has a future - including with his children - it will grow organically, especially if it is not forced.
Good luck, and keep in touch - we are always here to help, and there is plenty of experience on the AB to call on.