Per unit of heat ......... I think I'm right in saying that LPG is still the most expensive. It's big problem is siting the tank. The rules are horrendous, even to the point of having to put it underground if you can't comply with such things as distance from a boundary etc.. I'm sure you'll be getting a quote for this though.
Positioning oil tanks is also fraught with rules now, but not quite so onerous.
One great advantage is that gas boilers can, if serviced regularly, last for twenty years or more. Oil boilers are quite a bit more expensive to buy, and don't usually last anything like as long as a gas one.
Al mentions gas for cooking. Absolutely right. No need for electric cookers. In rural areas around here, a combination of the two is very popular. You have an oil boiler, but also propane cylinders piped to a gas cooker. (That's the orange cylinders about four feet tall. You have two of them with an automatic changeover valve, so that, when one runs out, the other comes in. The local LPG/propane guy comes around occasionally (even if you're out) and checks and replaces any empty bottle so you never run out.
Get full installation quotes for both systems, and take your pick.