The police can not only check your phone, but they can also obtain your phone records from your provider (they can obtain these even if you;re on pay as you go).
However, if you were not using the phone, I can't see the issue here.
Did they ever prosecute that guy who was eating a Mars Bar whilst driving a while back? I think the crux of the matter is being in full control of your vehicle at the material time.
It seems a little petty given the amount of people who don't drive with both hands on the wheel - I drive taxi style (ie one hand on the wheel, one on the gearstick), so technically I'm not in full control of the vehicle