Are you saying you have pulled out of a booking for THIRTY people. If you have I can see that's a big blow to the holiday company, hotel and airline and the travel agent must have put in a lot of work recording 30 people's details etc. and sorting accommodation etc .
So I think you should expect to have to pay something, and that depends on the contract.but maybe you should try to negotiate a lower payment. How much deposit inTOTAL are you being asked to pay?
Can you tell us what the contract said about cancellations?