This handy leaflet explains it all - including the rules for a self-contained unit.
http://www.voa.gov.uk..._understanding_ct.pdf
The issues that have blown it for you are:
the separate front-door
the facility for cooking (kitchen)
and I assume it has its own loo
I assume the annex has been given a Band A rating.
Your existing property will not necessarily have lost enough in assessed value to take it below the next threshold down - but it is worth a try.
If your existing dwelling is band F or G it is less likely to fall down a band than if it is Band D or E (because the width of the bands is much greater towards the top end).
The leaflet also explains