My neighbours often want a chat on a weekend when I'm in the garden to see how I am which can be nice for a few minutes but does hold me up. I often say, when I'll have to be getting on or it's getting chilly or I don't want to hold you up and that seems to do the trick though mostly I have to say it twice... You could always have a member of the family call 'Dad' or your wife call you after a while to which you say well, must go what would they do without me... or something to that effect.