You are indeed a patient lady, Mrs P.
If you are sure you are behind the curve on making payments versus work complete, then you could still reject his recent approach and just get the work done by someone else.
When you raising the 1st question, you indicated that the quality of the work was not an issue - just the timeliness. Now you are saying the walls and floors are uneven. Are you therefore expecting these to be taken up and redone? It is not easy to correct uneven plastering or screeding. Tiled surfaces that are unacceptable have to be taken up, scrapped and retiled.
What exactly has to be done (roughly describe it) to complete and how much money is unpaid versus the original contract sum?
The trouble with builders is they want a steady flowline of work. So some start more jobs than they know they can can complete in the time that they said, so they have work to go back to when the 'front-end' of the supply chain starts to dry up.