you sound ridiculous tbh - instead of just saying youre busy etc, you go straight to wanting to accuse him of harassment & come here for back up to accuse him!
is that really easier than just saying youre busy when he asks you?
why go for lunch or except lifts off him if youre then going to get annoyed that he offers?