Since your son's father does not have parental responsibility, he has no right to grant or refuse 'appropriate consent' for you to take your son out of the country.
Brief summary of the law:
http://www.ips.gov.uk/passport/apply-child-rul es-parental-responsibility-rules.asp
Full text:
http://www.statutelaw.gov.uk/content.aspx?LegT ype=All+Legislation&title=child+abduction&Year =1984&searchEnacted=0&extentMatchOnly=0&confer sPower=0&blanketAmendment=0&sortAlpha=0&TYPE=Q S&PageNumber=1&NavFrom=0&parentActiveTextDocId =211349&ActiveTextDocId=211349&filesize=58112
Of course, your ex partner is still entitled to apply for parental responsibility. If he acquired it, he could then prevent you from taking your son abroad (other than for short holidays)
Quote:
"A father can apply to the court to gain parental responsibility. In considering an application from a father, the court will take the following into account:
* the degree of commitment shown by the father to his child
* the degree of attachment between father and child
* the father's reasons for applying for the order
The court will then decide to accept or reject the application based on what it believes is in the child's best interest."
Source:
http://www.direct.gov.uk/en/Parents/ParentsRig hts/DG_4002954
Chris