Most the Medway towns are dumps but it's got one real jewel, Rochester:
http://www.visitmedway.org/destinations/rochester
It's got very few chain store but loads of independent traders,including secondhand bookshops, etc. It's admittedly not a place that you'd travel hundreds of miles to see (as you might with York, Bath, etc) but if you're in Kent anyway it's definitely worth a visit:
http://www.kent-life.co.uk/out-about/places/10-good-reasons-to-visit-rochester-1-4353354
While you're in that area make sure that you visit the Chatham Historic Dockyard:
http://thedockyard.co.uk/
[NB: Allow a minimum of half a day for your visit - It's big!]
You've mentioned the White Cliffs of Dover but make sure that you see them from the top. If you drive up to the car park on Langdon Cliffs (£3.50 if you're not a National Trust member) you'll find yourself in a unique place, where you've got beautiful countryside around you but you can look down onto the constantly busy port below:
https://www.nationaltrust.org.uk/the-white-cliffs-of-dover
Whitstable is wonderful. It's another town where there are far more independent traders than there are big chain stores. (It's 'upmarket' without being 'snooty'). There are loads of places to eat and drink and some great coastal views. (If you want a lovely sandy beach, with some nice tea rooms, etc, simply head off to Tankerton Cliffs, just to the east of the town. It's walkable).
All of the above is written from personal experience. One of the places I've not got to yet though (but everyone tells me is worth the journey) is the Romney,Hythe & Dymchurch Railway:
http://www.rhdr.org.uk/
It's worth taking your passport to Kent in order to take a day trip to France. If you book in advance, a day trip to Calais only costs £16 per person:
http://www.poferries.com/en/dover-calais
Calais is much-maligned by many people but it's got enough to hold your interest for a day
http://www.calais-guide.co.uk/
and it's packed with great bars and restaurants. I recommend the station buffet (which is absolutely nothing like a British station buffet - it has some superb daily specials)
http://www.buffet-gare-calais.com/
Au Calice also has great food:
http://www.lecalice.com/
Another trip to France (which is currently on my 'to do' list) is to fly to Le Touquet for the day from Lydd Airport. At £160 per person it's not cheap (and you need to book early because the aircraft they use only carry 9 passengers) but with only 15 minutes in the air you get plenty of time to explore Le Touquet:
http://www.lyddair.com/
You can also visit Paris, Lille or Brussels for the day (with fares starting from £69 if you book early) from Ebbsfleet International or Ashford International:
https://www.eurostar.com/uk-en/?
Getting back to Kent itself though, Hever Castle might be of interest to you:
https://www.hevercastle.co.uk/
or perhaps Turner Contemporary at Margate?
https://www.turnercontemporary.org/
(Margate itself is a bit of a dump but it's got a lovely beach)
If you're a fan of gardens, Sissinghurst is rated as among the best in the country:
https://www.nationaltrust.org.uk/sissinghurst-castle-garden
It's only 16 miles from Tunbridge Wells, which one the UK's most affluent towns:
https://www.visittunbridgewells.com/