I don't think so, no. It seems to just automatically email you.
The best idea would be to mark that email as "spam" or "junk" while checking your email with Yahoo. That way it should go straight into your junk mail folder, and not your inbox.
As joko says - go to the page your question appears on.
To the right of your question is a box with a red header saying "Got an Answer?" - if you click on the "click here" bit , you should stop receiving further "answer notification" e-mails for that particular question.