There are things about this question I just don't understand.
You are going to have to describe it a bit better.
For example, in one sentence you say 'the driveway is directly attached to my neighbours'. Then in another sentence you say 'they (the council) own a small bit of land between our two driveways'.
Was your discussion with the council perhaps about the ownership of the land outside the front perimeter of your land, where it would be perfectly correct that the council do own the land (the verge and the footpath - if there is one)?
What prevents you widening your part of your driveway (on the other side to that which abuts the neighbour) to enable you to get your car in and out?
Unless there is a statement written into your original land title that enables you to have vehicular access over part of his driveway, then that wasn't the original set-up.
If you have been using part of his driveway as part of your access to your property for 25 years, without formal permission, then you may well have acquired a permanent right to continue to do this (an easement) under the Prescription Act 1832.
However demonstrating your right to this and enforcing it legally, including getting it written into both your land title and his at the Land Registry, is going to cost you money because you would have to employ a solicitor to advise and help you.
The same is true of him on you (though it appears he does not wish to invoke this opportunity if he is building a wall down the middle).
But before we go any further with this, please can we clear up this business about which bit of land may be in council ownership.