If it's as you've said here then I would write the LL a letter explaining that he has already checked the room and found it to be ok. The LL can't charge for fair wear and tear anyway and what you've described here would, at worse, be fair wear and tear.
Also, if the deposit is held in the DPS or similar then it's only the sum of money for any damage to the mattress that would be withheld initially whilst in dispute. The majority of the deposit would be returned straight away.
From what you say though, it would be very hard for the LL to prove any damage unless he has strong evidence of before and after condition of the mattress.
Write to him, explain that you will contest any deduction and I expect he will back down.