Never mind albalass, you are not alone. I'm from East Anglia and nobody ever recognises that accent (really accents, because we can spot slight differences). I've known it mistaken for Australian !
In France my own French is accented in such a peculiar way that people in Geneva sometimes think I'm from the South of France but people in the South of France often think I'm Swiss or, sometimes, Belgian. Pity, because I've had a home in the South of France for forty years. That shows how subtle the ear is in natives.They know something is not quite right but struggle to place the accent because it's not consistent. Generally English people who live in France can only identify very marked accents found in their own region but not other French accents. The French themselves find the accent of North East France quite hilarious but nobody English I know has ever been able to spot it.