If you don't want to give them your address directly, you will have to put the matter in the hands of your insurers.
That way, it means of course it would affect your no claims, however that's not the end of the world and it would at least mean that you wouldn't have to deal directly with the other party, particularly the partner who sounds as if he's after as much as they can get, even if he doesn't actually own the car.
If you want to deal with it yourself, then the other car owner is entitled to your name and address. However, I would only have dealings with whoever owns the car, not the partner, it is nothing to do with him if he does not own the car.
Weigh up the options once you've had their estimates, plural, then decide finally what is the lesser of the two "evils" from your perspective.
Good luck.