My apple and pear tree's have given me a reasonable crop in their 2nd year, when is the right time to prune the trees as they are looking a bit scraggly.
I also had a plum tree but have now destroyed it as it had Canker. I thought it easier to replant another than try to cure it as the base of the tree was splitting.
Personally, I'd wait a little longer til October or November, when the tree is in a more dormant state.
Prune branches back to a fruiting bud + remove any dead, damaged or diseased material.
Ps, I think yuo did the right thing with the plum.