The OR is saying you have been given preferential treatment when compared with the other creditors, and this is something the OR has the legal power to undo by demanding the payment back from you, however unfair that may seem to you.
Your original post says your brother paid you in full after selling his house, but that he had other creditors he could not repay and tried an IVA. Was he in the IVA when he repaid you? If so, he must already have been behind with his full payments.
If he was not in the IVA, are you absolutely certain that when he repaid you all his payments to the other creditors were fully up to date AND that he did not have any reason at that time to think he would be unable to go on making the full payments to them. If this is the situation, it is just possible you may have a case. You also need to know what your brother has told the OR about all this.
The OR has to apply the law, but does have some discretion in reviewing the circumstances of individual cases so it could be worth putting all your facts together and asking in writing for the decision to be reconsidered. If this is not successful you can, if you want, ask for it to be reviewed at a more senior level but do bear in mind what Golden Shred says about the possible cost implications.