I don't bother (other than to have a very small amount of cash in my pocket). Although you get charged for using overseas ATMs (or for paying food goods and services directly with your debit card), the much better exchange rate you get that way usually negates those charges.
However the best High Street rates are often obtained through pre-ordering. For example, you can get a better rate at the Post Office if you order online instead of just walking up to the counter:
https://www.postoffice.co.uk/foreign-currency
I agree with the other post regarding using a card, I have Nationwide cards and as long as I remember which does what I pay no charges on purchases or cash. However if I'm buying currency I've used IMS FX in the past, it depends on what you call a modest amount, over £500 there's no additional charge.
We always use the Post Office, mainly because of convenience but its rates are usually not far from the best anyway. John Lewis (according to an email they sent me) will match any price you can find.