''I don't know who they are '' They carry official photo ID which they have to show you.
I really do not see what your problem is.
You fill in the declaration that your address does not need a TV licence and what the reason is. Once you do that they stop sending letters about a TV licence. They still have the right to visit you to check that what you say is true. If they visit, you just let them to look round, you will be there while they are inspecting your house. They cannot enter unless an adult gives them permission, so they are not going to try to get in while only a child is present.
A man I work with does not have a TV set so does not need a licence. He filled in the declaration 3 years ago and renewed it last year. He has not had any reminders since he filled in the 'no licence needed' declaration
(apart from a reminder to renew the declaration ) and he has not had a
visit yet.
Before he filled in the declaration he used to get letters all the time and was always complaining about it.