Sounds like you've made a form of raised bed, in my book, the best way to grow vegetables in the garden, all I can do, is tell you what I do with my raised beds, and I get very good quality veggies every year.
All I do, is put a layer of Horse Manure on top, (encourages worms, apart from the natural goodness of the stuff), then cover the lot with a thick tarpaulin, helps keep the heat in, aids self composting, and come the new planting season, approx May time, its ready.
Although I would change the compost the potatoes are in, or use a rotate your crops.
Hope this is of some help.