As others have said, and I can endorse from my own experience, buying your first property is very stressful and very scary because it's probably the first really big responsibility most of us take on in adult life and the penalty for failing, either in the relationship, or defaulting on the mortgage, is awesome to contemplate.
So try and understand between the pair of you whether your relationship has actually reached its "sell by" date and it is the thought of the high price of failure that is bringing you to this point, or whether it's just the general anxiety of it which is preying on your boyfriend's mind. It certainly won't get any easier once you have the ongoing commitment of a mortgage, so you must sit down quietly with him now and be honest about your feelings of doubt. Perhaps his reaction, (either shock, or relief) will give help you decide whether you should still go ahead, or pull out now.