A V5 document has nothing to do with the legal ownership of a car. It simply states who the 'registered keeper' is. For example, most people who drive company cars will hold a V5 in their name, despite the fact that the car is actually owned by their employer or by a leasing company.
By accepting your old car, his man has entered into a legal contract to transfer ownership of his old vehicle to you. The V5 has nothing to do with it. However, if you don't receive a new V5, you're meant to write to DVLA to tell them about it. Under the circumstances, it might be a good idea to do so as soon as possible. Similarly, if you really think that he's likely to steal the vehicle, you might decide to have a word with your local police station, to make them aware of the situation. If they're given advance notice of the position, they'll know that they should treat the removal of your vehicle as theft, rather than as a civil dispute. (They might also decide to 'have a word' with the guy, regarding his threats).
Chris