The legal position would be that a deposit is a sum paid by one party to a contract to the other party as a guarantee that the first party will carry out the terms of the contract; the first party will forfeit the sum in question if he does not carry out the terms. If the contract is carried out without dispute the deposit becomes part payment.
It is useful to understand the legal position but it is not always the best way to proceed, your moral feelings may be a better judge of what is fair and equitable.