One explanation sometimes offered is that the Vikings were - according to legend - guilty of slicing people's noses open during their raids on Britain 1000+ years ago. That is, if the victims failed to come up with the tributes/ransom-money etc demanded, they would have their noses slit as punishment.
That's not very likely to be the explanation, given that the earliest use of the phrase in English dates back only as far as the 17th century. The problem with the rhino = money as well as nose idea is this...why do we say 'through' the nose? We wouldn't talk of paying 'through' the money, would we? I think 'origin obscure' is probably the best answer.
AB Ed: Elsewhere, Tartanwizard added: "I think it refers to extortion, when if a person didn't pay up they'd get a punch on the nose and pay for it in blood. So to pay through the nose for something implies that the price is 'extortionate'. One of those exaggerations often used in everyday speech. "