Always look in the 'bargain bin' bit of supermarkets, they sometimes have nice stuff there. That as well as places like �1 shops, Lidl and so on which have European stuff at a fraction of the price of stuff in M&S and so on.
To start with re: yummies, there's always crisps and nibbles from the above, then cut up some carrots (cheaper than pre-cut) for dips and so on. You could always make some fancy ice cream by buying a good supermarket version and adding your own ingredients and making it Ben & Jerry's style with bits in it.
Good luck!