There are many things that distract car drivers: passengers talking, music (I'm sure I read somewhere that fiddling with car stereos was a huge factor in road accidents), the necessity to watch what the "I know my limits and those of my car, so I will drive at the speed I feel I am entitled to" brigade are up to, children playing up, smoking, smoking hot ladies on the pavement etc etc.
I totally agree that talking on a phone is a distraction, but cannot see how it is a worse distraction than any of the above. A person in the car with the driver will demand more of their attention than one on the phone; one can always turn the phone off if necessary - that doesn't work on the mother-in-law.
As for the business aspect, things have changed - we all expect deliveries next day and even in a certain time frame - so either we give that up, or we allow some form of communication to drivers. Thinking about it, cab drivers used to use those hand-held mikes all the time without causing mass road carnage. Maybe it's the drivers rather than the talking...