Well, from what I understand of 106 Agreements, they were devised so that people on lower incomes could also afford to be part of the property market. A client of mine is in a similar situation to yourself and unfortunately it is having a negative impact on the sale. I hope that it is different for you! In my clients case, the Section 106 Agreement instructs that when she decided to move, the local authority must offer the house to the people it was intended for ie people on a waiting list for such properties, if no-one can be found for the property she was free to put the house on the open market however it does restrict the purchaser when they wish to sell in th off chance that a suitable candidate is found for the house. Which needless to say, then means that they will not receive anywhere near the amount paid for it.
On the plus side, one of our conveyancers is trying to bring this up with the local authority, so quite possibly they may have to rethink their actions, or perhaps revise the conditions of the 106 Agreement.
Apologies if I haven't been very clear!