if you send an encrypted e-mail ... how do you expect the recipient to decrypt the message unless you give them access to your key.
If you've done that ... of course they know who you are.
the second part of your question
depending on the level of encryption ... almost all cyphers can be broken given enough time ... the question is are you worth the effort?
as for can you be identified .... of course you can ... your ISP must by law keep records and even though your mail address may be anonymous it's your connection that generated it
every packet you send carries both destination and sender address' (how else could you get a reply?)
and when you register an anonymous e-mail address ... the ip adress used is logged - also whenever you send
that's how spam works
how do you think they catch"cyber" criminals?
again the question is ... are you worth the effort?
Generally AC they don't (catch cyber criminals - only the dumb ones)
These sort of people don't set up an account with their frindly local ISP who logs their details, they'll send mail via open relays, compromise other machines and proxy through chains of other computers.
I'm rather assuming that Harry isn't a budding cyber criminal but is either intent on a practical joke or wanting to send a romantic mail or something of that sort and is confused between anonymous and encrypted.