Is this a shared driveway, where the physical land ownership is split roughly down the middle, but where (in your legal land title) you both have a legal right to drive over the respective part owned by the other? It is often indicated on the title plan by hatching the area of shared access in a colour - often brown.
And if so, has the neighbour now removed part of your 'garden' on your side that was not part of the shared driveway, and paved it?
If so:
yes, it is likely to cause a problem when you sell the property - because the hatched area on the title plan patently no longer aligns to the area on the ground.
no, you cannot lose ownership of the LAND strip.