I don't think it has any particular origins, perhaps as Ed suggests, or perhaps you use hole number one for number ones and so on.
It is a euphamism for replacing the more vulgar words, with phrases that can be used in polite company - such as with children.
There are any number of lengthier periphrases for excretion used to excuse oneself from company, such as to powder one's nose or to see a man about a horse (or dog) or doo-doo's etc. Slang expressions which are neither particularly euphemistic nor dysphemistic, such as take a leak, form a separate category.
QM may correct me on the route use or first known usage if there is any historic record.