I think they're generally solitary, so it's a bit academic.
If there is a "proper" word, it'll be a one of those made-up ones like a parliament of owls. I doubt if anyone ever saw a group of lynxes and genuinely needed a word to describe them.
It'd be just as valid to make up your own. What about a pounce? Or a flurry?