Probably the only thing you could do is to apply to Court for a contact order. However, the Court has to follow principles which do not necessarily include giving grandparents contact rights if this is opposed by the parents (or one of them). You would need to take advice from a family law solicitor, and it might be expensive to pursue to Court.
You also need to bear in mind that doing this could make arrangements with the mother even more difficult than they are now. She could even stop your son seeing his child and Court action would then be needed to try and enforce his rights.