Probably lack of pollinators, you could try planting some bee friendly plants nearby, you might get Echium and Limnanthes to flower if you show them now. Try spraying them in the evening with plain water, and feed and water well. They are really greedy plants
i agree with Rowan, a lack of pollinators or maybe the wrong type of pollinators, ie robber bees stealing nectar from the base of the flower, due to their short tongue (proboscis). growing plants to attract a wide range of pollinators seems to me, the best way forward.