For "dirty" read "mouldy". The dampness from the water (which also goes in!) causes mould to grow.
Take the dispenser drawer out and clean AND dry it after use. No more mould!
remove then soak it in a bleach & washing up liquid solution for about 30 mins then rinse, do this once a month and" every little thing gonna be alright".