I think it's definitely an issue if you are made responsible for losses from a till which, if it is usual practice for others to use it, is not solely under your control, especially if it's management doing it. Have they asked the other managers about this? It's not quite the same as you letting any staff dip in and out of your till.
You could point out that there needs to be a clearly defined and adhered to policy to safeguard honest users and stop any abuse of the system. If you can't have control over your till then is it fair to penalise you for losses from a till you do not have control over?
Unless you are allowed to assert authority over who uses your till while you are on shift it seems unfair to make a user solely responsible for any losses which cannot be directly attributed to them. Are there any other policies in place relating to who can use tills which might come into play?
One place I worked, a restaurant, we had sole responsibility for our cash (we carried it in bumbags) and it would be counted out in front of us by the shift manager at the end of each shift. Another place, supermarket, they would sometimes take the till to be cashed up after a change of staff or in case of a query - I had someone try it on saying I'd short changed them once but they took the till and counted it there and then and it was spot on.
Have they checked the amount against other tills to rule out money being put elsewhere ie if another till is £5 up. Definitely worth checking.