Yes, ideally the agents should have contacted you first to say they were going round with the buyers to measure up etc but I'd try not to get too upset just because they didn't ring. It's just the way a lot of agents are. They probably didn't think anything of it - and, under normal circumstances, no-one would mind, especially as the buyers are moving in in a few days time.
A few pieces of carpet lifted at the edges is really here nor there. I know it feels like an intrusion at the moment, but it really doesn't matter in the grand scheme of things.
I'm not being insensitive, but I wouldn't go overboard just because the agents were a bit remiss about not making a telephone call.