OK, I admit I'm a fella', in his fifties, who knows next to nothing about relationships, so I might not be the best person to answer this.
But I do know that working next to someone who wears perfume (even if they're drenched in the stuff) is unlikely to leave more than the slightest traces of perfume (if any at all) on your clothes. The only way that the odour could be marked would be if there was contact between two people's clothing (or, of course, their bodies). If the scent was only noticeable on a single occasion it could have been present through nothing more than an innocent cuddle, while (for example) people were 'larking about' at work. But if the scent was there on a regular basis it would seem likely that there was more intimate contact taking place.
That's probably not what you wanted to hear, and it's a rather 'forensic' answer (rather than a 'caring' one) but you asked for opinions, and that's how I see it.
Chris