Hey Feelinlost
Try not to worry - even if the GP did ask, he would be unlikely to then double-check your own records.
If you're worried, you could phone the Dr before the appt and advise of the situation.
I took your question to mean that you (or rather, your partner) are not actively trying to conceive...even if you were (without you taking contraception, obviously), there could be any number of reasons why you wouldn't be pregnant - eg there could be a problem with you, or it could just be nature. Not everyone finds getting pregnant easy!
It will do your partner no harm to have his sperm counted - if it's fine he should be reassured, and if not then at least he knows.
It's not a great sign for your relationship if you're lying to him re contraception, but at the end of the day it is your choice, not his.