That's true. But Parliamentary Privilege seems to apply in this case.
Hoyle's full response was "I will not intervene unless something is said which is disorderly... such allegations should not be made lightly, especially in view of the guidance about good temper and moderation. While they may not have been disorderly, I am far from satisfied that the comments in question were appropriate. I want to see more compassionate, reasonable politics in this House, and this sort of comment can only inflame... I want a nicer Parliament... we should be more honourable."
It seems pretty obvious what Hoyle thought of the comments. He may not have demanded an apology, because he felt he could not within the rules of the House, but he sure as hell wanted one.
https://www.mirror.co.uk/news/politics/breaking-speaker-lindsay-hoyle-slaps-26106857