How did the issue come up, was it raised in the valuation, probably done for the mortgage lender.
If so then your lenders may well have a say in what can and cannot be done. It should be checked if they are satisfied with indemnity insurance.
It may have come up from a local search if planning was noted but no building regulation approval or revealed in information forms filled in by the seller or replies to your solicitor's enquiries.
Solicitors only tend to be overly concerned about lack of building regs if they are under 10 years old (esp under a year as indemnity insurers won't cover it) as it's generally thought that it's past any enforcement action by the council.
Regardless of that I'd want to make sure there is not issue with the wall ie that it was done properly and is not going to cause you a problem either during your occupation or sale eg the wall hasn't been erected properly and could fall down.
Indemnity insurance, payable by way of one off premium which is usually based on the indemnity limit required - usually the purchase price but some lender's require a specific amount eg a percentage above their mortgage advance or full market value) is common but to be honest almost too common these days as it's seen as a cover all solution to any problem BUT you need to consider what the insurance can cover, the terms, conditions and, more importantly, limitations or restrictions. It's an insurance policy not a magic sticking plaster.
For example, in most cases, an approach to the local authority to check things out can invalidate any claim as, as the insurers will see it, you have put them more at risk of a claim by alerting them.
There is a growing habit of just putting in a few details online and downloading a policy without proper advice and consideration being given so please make sure everything has been checked out properly.
Hope