Tell your neighbour that the fence is dangerous, it has already injured your dog, it is an eyesore, that it is trespassing on your land and give him 5 days to dismantle it.
Remind him that you had already advised him you would have a proper fence installed as soon as convenient and that will continue to be the case. However, you are not prepared to put up with a shoddy home-built job as that will be a permanent eyesore to you, and will probably be a negative in selling his property. If he won't dismantle it himself, can you appeal to any neighbours to come and help youi, and make sure you put all the rubbish back in his garden to dispose of. If they're moving house soon, you won't have the embarrassment of having to face them much longer, so don't be intimidated about it.