There is abolutely no point in making a point about giving him a low-key birthday because of his behaviour, unless he is absolutely aware of why you are doing it.
That means a discussion. If you were, and obviously are upset about his behaviour, it is important enough to be discussed properly as adults - no use saying yo 'don't communicate' because that is simply absolving yourself of the responsibility for acting like an adult in an adult relationship.
If things are that bad that you feel you cannot deal with this properly, you should be thinking about your future with this man, and that of rouy son, and asking if he is really a suitable partner and father-figure for you both.