I am sure it is ok. I used to take my shoes off to have driving lessons (I kept my socks on) and my driving instructor told me he was wondering, so he checked with his branch supervisor (BSM) who confirmed it was alright (this happened in 1999).
I once put this to a Police friend of mine who said it may be seen as not being in full control of the vehicle??? I drive in bare feet if wearing "heels" because I am worried that they may hinder my driving but I was told I should have a change of shoes in the car which would enable me to drive. Another point to ponder is if you were involved in a collision that extra bit of protection to your feet may save them.