I know that everything about me is stored on computer somewhere. Apart from the Government and associated bodies who is legally entitled to keep my details on record? For example, can the local Parish office of the Church keep all my and my familys information legally or can I insist and see evidence that those details are removed?
I believe that people can only keep information about you legally for as long as it's necessary - e.g. if you apply for a job and don't get it, your application and any ID should be destroyed. I'm sure that anyone keeping information must keep it in line with information governance legislation and that (unless it's for reasons of national security) you can ask to see what they have on record for you.
Why would the church have all your information, anyway?
Parish records are only of marriges and funerals that have taken place in the church concerned. They were traditionally kept in a hand written ledger but I suppose they have a computer record now.
The ancestry sites have wedding details going back hundreds of years. More trouble than its worth to start contacting all and sundry to have your details removed.
I used to work in Data Protection, the others are correct, that information should only be kept as long as necessary. Church records fall into the public domain and no such requirement exists to have that information removed. The main reason would be if you wanted to keep getting married and wanted to hide you were a bigamist or denied you had been married before etc etc.