Almost certainly not.
Your question seems to have (at least) as much to do with the use of the English language as it does with contract law.
People say things like "I may go shopping tomorrow or I may not". That (incorrect) use of English treats the word 'may' as a synonym for 'might'.
People also say things like "I've told my children that they may go to the party". That (correct) use of English uses the word 'may' as a grant of permission.
If I've read your post correctly, you're basing your argument upon the former (incorrect) use of 'may'. If you've entered into a contract which states that the other party 'may' take money from your account, you seem to have granted them permission, since the latter definition is the one which applies.
If (as seems unlikely) you've entered into a direct debit agreement, you can cancel the payments with your bank. (You could also seek to get a refund on any payments which you believe might have infringed the direct debit code). However if you've agreed to a regular payment from a debit card you can't cancel the payments. Your bank will only recognise instructions (or, rather, the lack of demands for payment) from the payee.
Chris