http://www.well.com/user/woa/fspot.htm
That is a fact sheet that claims that marijuana does affect driving. Anyone who has used it and is a driver knows it does impair driving ability. When you are young and selfish (in my case my late teens), all I was worried about was being stopped by the police, now old and sensible I realise it would make me a risk to others on the road and anyone who was a passenger.
Whether you want to respond or not to your neighbour is your business, (perhaps you could print off the fact sheet and highlight the bit about driving and put it through his letter box!), but yes his driving ability was affected - how much is debatable depending on how much was consumed. Lots of regular drinkers of alcohol reckon they can drive after one drink same with cannabis, people reckon they now their limitations. Most cases they are still adversely affected whatever they think.