Based on what you have said, at the very least there should be a written warning for misconduct here.
If the company does not do that, it severely undermines your position with your colleagues and your role in the organisation - afterall, everyone is going to know about this in the company.
Then there is the issue about whether you can reasonably be expected to continue managing him. This is more difficult (for the company to deal with), but I would advise discussing with HR the possibility of him being moved to another department. This would also send a supporting message from the company to you, in the eyes of others in the company. It may not be possible - depends on too many factors to advise.
If the above is not possible, depending on how you feel about it, you could raise with the company about his dismissal. Much will depend on how he is perceived, how you are preceived. If they don't value him much but you are highly valued, the concern an HR department would have is about claims for unfair dismissal. Of the various 'fair' reasons for dismissal, 'misconduct' could be used, on the grounds that his conduct has resulted in a total breakdown of trust between him and you, and the company has no other alternative role for him.