It comes from an old engineering saying to do with steam engines:
The meaning has to do with the centrifugal governor on a steam engine - the
weight was called the mouse, and as the engine rpm increased the mouse
would rise due to centrifugal force. But as the mouse rose, the arm
would force the steam valve in the more closed direction, thus
reducing the rpm, that is, the higher (the mouse), the fewer (rpms).
So for a given setting of the mouse on the arm, the engine would run
at a constant speed.
So the riddle should actually be: How is a mouse when it spins? - The higher, the fewer.
Obviously due to the fact that this riddle is so old it has been open to misinterpretation over the years.
I cant take credit for this detailed answer quizmouse, I found it via google.