Poor you. We had this problem for about three years and tried everything; we were at our wits end.The little blighters go for the expensive fabrics first, don't they? In the end we had to board out our pets and pay for professional exterminators to come in. This was a while ago and I can't remember who did it but it worked.
I still have lavender gel hanging in the wardrobes and all the drawers are full of moth protection bags. I am totally paranoid if I spot anything small flying about.
Good luck.