One of mine will only go on grass, which we have removed and replaced with slabs, and she has to be taken out for a walk. So I dont think it is always to do with being frightened of going in the garden. I dont take her a walk late at night, she goes out for a long walk twice a day, early morning and late afternoon, and what she doesnt do then she holds until the next day. All mine spend a lot of time down the garden, even late at night, but she wont do anything down there, her choice.