You're on the right track but it depends what you can get hold of. Try 50% (V/V) hydrochloric acid and leave it overnight if necessary. Nitric acid at the same strength would also do the job.
Avoid any alkaline substances such as caustic soda as they will affect the glass.
But to be honest, rather than mess about with acids, I'd mix up a strong solution of biological washing powder in fairly hot water and leave it overnight in the bottle. This might well do the trick.
My lab technicians would funnel in piranha acid or chromic acid into the bottle in a fume cupboard. The deposit would be gone in seconds. Oh the fun we have in laboratories!
Seriously though, if you do decide to use acids, make sure you don goggles, gloves and observe obvious safety precautions.