There's nothing you can do about it. As long as he avoids contravening legislation regarding such things as racial discrimination, any trader can charge what he likes, when he likes and to whom he likes (or dislikes!) without giving any reason.
For example, persons A, B, C, D & E could all be in the travel agency at the same time. The travel agent could sell the holiday to person A for �1150. He could then refuse to sell the holiday to person B at any price. He could offer it to person C for �1. Person D could be told that it would cost him one million pounds to buy the holiday. Person E could be told that he'd be allowed to pay the 'regular' price of �1150 but only if he agreed to sign the contract while standing on his head, completely naked, and whistling the Slovenian national anthem backwards. The seller sets the price and conditions; it's up to the potential purchaser to decide whether they're acceptable or not. You decided that the price was acceptable to you, so you're committed to going through with the contract at that price.
Just enjoy your holiday ;-)
Chris