When I split from my son's father I was concerned that if anything happened to me he was not a good person to raise my son.
I made a will and stated that it was my wish that my son be taken care of by my parents. This was perfectly legal, but the soolicitor did warn that as his father has equal parental responsibility he could contest it.
OI would imagine you are in a stronger position as your children have already said they don't want to continue contact.
However, this was some years ago now, so you may want to check with a solicitor
:)