jason, your barrister was right; it is potentially a double-edged sword. If they didn't hear the noise themselves, it is hearsay, and should not have been adduced. If it got in, the judge would have told the jury that it was not evidence and should be disregarded. So would counsel.
That being so, it being hearsay, it suggests that the witnesses are desperately trying to make the case worse, and , as such, merits strong comment. If it were not hearsay, it is something not to dwell on, because it suggests malice on your part and fits the alleged racial abuse. That could be said of hearsay too, but played right , the jury will be told to ignore it. And you need not be asked about it.