Yes we should return the marbles to Greece as they were effectively stolen in the first place. The British govt may have purchased them in the 19th Century, but they presumably did not do so from Greece.
Having said that, the Greeks have never formally requested their return: they have used the issue for political posturing, much as Spain does over Gibraltar, although admittedly in their case with more justification.
It is also not true to say that Britain has said they would never be loaned out. It was simply that no one had ever asked. They claim they would lend them to Greece only there is no likelihood they would be returned.
Meanwhile I see no reason why a single statue should not be loaned to the Hermitage. It's pretty high profile, so presumably there is little chance of the Russians stealing it and offering it to a FIFA delegate as a bribe, as happened with other Hermitage artefacts during the World Cup bid