aside from the fact that you shouldnt have to put up with this.. if you do have the money to spare - get yourself a dehumidifier.
I bought one for £89.99 from Homebase recently for a similar problem - the water vapour in the air is the issue.
It collects loads of water - and is very quiet
you will probably need to wash ALL the clothes in wardobe to ensure you have caught it.... Dont cross contaminate areas either or it can spread!
I found this link useful - though its in US
http://www.doh.wa.gov/ehp/ts/IAQ/GOT_Mold.html
Winter will be worse as its caused by warm air meeting the cold. Windows collecting condensation are a tell tale there is a problem with ventilation
Hope this helps