The jewel in East Anglia's crown has to be Southwold. It's a delightful town, full of great pubs with good food, and with a superb beach:
http://www.exploresouthwold.co.uk/
Southwold is just a few miles off the A12. By public transport, take a train to Halesworth and then bus no 521, from Halesworth station, to Southwold. (Buses are timed to leave shortly after the train's arrival).
Otherwise, North Norfolk has plenty to offer. If you want somewhere which isn't too quiet, try Cromer:
http://www.visitnorfolk.co.uk/norfolk/cromer-s heringham.aspx
Cromer is served by road and rail from Norwich.
Chris