Replace the term "withhold" with "commit theft of".
The passport is not the property of the father (it actually remains the property of The Crown) - he therefore has no right to retain it and to do so is theft.
Either report the theft to the police in order to obtain a crime number for the LS01 lost or stolen declaration form or complete the section "Other" with details of the illegal retention (this may prompt the IPS to investigate further - identity theft being such a hot topic). The LS01 accompanies the application for a replacement passport.
http://www.ips.gov.uk/passport/lost.asp