I reckon its root suckers from them plum trees, they can send them up from some distance, especially if the trees were grafted on to a vigorous root stock which is the usual case.
I have these suckers coming up all over the place in my garden from the original trees that were planted around 1855, thats why I reckonised them instantly from your pics.
If they are allowed to grow, they could form a thicket of thees, so the best advice that I can give is cut them off while they're small, just snip em off at ground level, they are very shallow rooted trees.