Hmm, he's obviously not yet on the same wavelength as you.
Maybe try the tack - 'further cracks have appeared in the structure which you surveyed 3 years ago and indicated that it was sound. I am considering the extent to which your practice provided me with sound advice at the time. In order that we might resolve this matter, please could we meet up at my property, such that I may hear your professional opinion on the circumstances. I am sure that you would wish to inspect the current situation in order to be able to assess for yourself what has happened here '
Note that this isn't threatening legal action or alleging liability. Might get him around for free. He answers to a professional body (who will be underwriting his professional indemnity insurance) so if it is still proving elusive you can drop into the conversation that you might need to talk to them if he won't come around to assess the situation.
I suspect he is will say it is normal settlement - at least you want him around to get him to say this.
Cracks can look quite nasty in appearance without necessarily being a structural issue that needs remedial work.