By the token that there is a disciplinary procedure, there also ought to be an appeal procedure. The appeal procedure ought to involve a manager (normally of a higher level - not necessarily the manager of your manager) who can look at the PROCESS by which the issue was dealt with, and whether the outcome seems reasonable given the evidence. Note that you can't bring up new evidence. Ask to use this.
You have given us the bare bones of the case and your side - and there are always two sides - but personally I can see that the loss of a safe key might be seen as more of an issue than the key to the building. Not having signed a letter regards being responsible doesn't count, I'm afraid.