Firstly, you'll have to go to the police station, either to pay up, or to contest, so tell them the error in details then.
If you truly have done nothing wrong, I would contest it. Data can be taken from the phone to tell if it was being used around that time. A friend of my husbands is going to court soon for this, she swears she wasn't using it, and it they have taken it for forensic testing.
If you were using it, or thinking about it, pay up and take it as a warning. I have points on my license for this, like you I wasn't actually using the phone, just holding it, but still...if it had rang, I'd have looked! I never even saw the police car, so God knows what else I could have missed!