It might be a loose brake shoe inside one of the rear drums Fluff. Sometimes the retaining clips or springs break or come loose which leaves a shoe to grind on the inside of the drum. It might also be that one of the rear brake cylinders has started to stick. In that case, a brake shoe might still be pressing against the inside of the drum after you have taken your foot off the brake and it grinds when you reverse the direction of rotation of the brake drum.
It could be something else of course as your description isn't very detailed but it does sound like the brakes are getting worn. I think your first move would be to have the rear brakes checked at a service centre such as ATS or Kwik Fit. Hope that helps.