It is highly likely that you have a hornets nest in your roof void, you can usually spot the hornets access to the loft just by walking very slowly around the house keeping any eye around the gutter/fascia area, that is where they usually go in. I suggest you do not go into the loft looking for them, whilst they are docile they will be attracted by a torch light as they are attracted to the lights in your house.
An alternative is to ensure you keep your windows closed in the evenings when they are so attracted to the light and tolerate them.
They are just trying to survive the same as the rest of us, but if they are a danger they must go!
They have no legal protection, so don't worry about that.