Old English sometimes altered words by changing the vowels. This is still the case in quite a few verbs eg I run becomes I ran rather than I runned. With plural nouns there are just a handful: foot, tooth, goose (oo becomes ee), mouse and louse, man and woman. There are a few where the plural doesn't change at all (sheep, deer) and a few others for historical reasons - ox becomes oxen, child becomes children (that's actually a double plural: childer+en).
These are all fairly basic words, or would have been among peasants a thousand years ago, and perhaps it was regular usage that stopped them defaulting to the more usual forms in modern English. As QM says, the more modern sort of mouse is sometimes mouses in the plural.