I listen to a lot of internet radio. (Usually many hours in a day). I've noticed that most web browsers seem to exhibit memory-hogging problems when listening for extended periods. That can result in the symptoms you describe. (So it might not be the time of day that's relevant when you get those 'sudden, brief silences' but the fact that you've had your browser open, and downloading audio content, for an extended period).
Periodically dumping memory from your browser can help. (With Firefox you need to open a new tab, so that you don't interfere with your listening, type 'about:memory' into the address bar, hit 'Enter' and then click on 'Minimize Memory Usage'. That might also work in other browsers but I've not tried it).
However actually closing your browser (when you're not listening to internet radio, of course) and then re-opening it is more effective.
Better still though is to use a web browser where the problem simply doesn't occur in the first place. Unfortunately that seems to rule out all of the most well-known browsers. I know of only one web browser were I can leave internet radio playing for hour after hour (or even for day after day) without experiencing any problems. (It's the one I'm using right now to listen to an old episode of Round The Horne). If you use it for listening to internet radio, irrespective of whether you use it for anything else, I suspect that your problems will disappear:
https://vivaldi.com/