What you describe is often a type of acne- a very common one even for people who dont get it anywhere else.
The best thing is to wear a loose top and go and show a pharmacist, who will be able to check and recommend. we did the same thing for my OH who didnt want to see a doctor.
If you dont want to do that, the main thing is not to pick at them! Try to let them heal, maybe apply tea tree oil or a spot treatment to dry them up, then use a SMALL amount of HC45 cream which you can buy over the counter which will aid healing.