They're all kind of right.
It's just a joke between the disciplines though; each of them think they're superior to the others.
(Disclaimer: I'm a physicist, so clearly physics is the answer to everything...)
Physics is what I think to be the fundamental science -- how everything works and interacts. Physics of the atom, specifically including the outer electron shell, and even more so longer molecules (organic), is such a large topic that it branched out into its own subject -- chemistry.
Biology, at its heart, is explained by chemistry (just as chemistry is fundamentally explained by topics of physics -- quantum theory, etc.). But it's more than just that, otherwise they'd still be chemists. They do things in ways chemists don't, for particular chemistries. Hence the separate discipline.
Maths is regarded as a science too. I myself see it more as a language to describe things more precisely than everyday language can. As a language, it's required for physics, chemistry, biology, everything. Anywhere where languages like English aren't good enough. There are those that work to extend the language -- mathematicians. They also look for patterns in the language that may reveal things about it we don't presently know. It follows scientific principle.