TTT - //I can't really nail it down but when you have a passenger that person is actually psuedo driving too and they react accordingly, you don't get that with someone on the phone. //
I think that is definitely something to consider.
When you have a passenger in conversation - even if it someone who doesn't drive - they can clearly see when the driver is concentrating, or working their way through a difficult junction, and so on, and they will instinctively sop their dialogue until that time passes. Obviously the person on the other end of a phone has no such idea of what is going on, and will blithely carry on, at a time when the drive may just need his or her faculties to carry on driving safely.