Abbeylee/ If you don't want to get involved with this older man, you have to make it perfectly clear that the friendship can only be platonic - maybe he mistakes your friendliness for something more. However, if someone has described him as a "man's man" then he probably would prefer to be down at the pub with the lads, watching sport, etc.
Don't be misled, that type like women too!