if the mole grip or pliers dont work try softening the plastic with a heat gun so that you can put it off of the pins, then pull them out individually with the pliers.
You may have had little luck getting them in because whilst plaster allows you to push/hammer into it, brick is less forgiving. It'll bend pins and divert the pin's trajectory. Were you successful in getting it in, the brick would probably crack. You need to be lucky enough to find a mortar joint. Or use smaller pins and trust the plaster only depth (risky).
Yeah, pliers or mole grips, and lots of wriggling/pulling.