That should be OK, IMO. How could he hold you responsible for damage if it's parked on a open driveway, to which everyone has access?
You ought to have a reciprocal arrangement, too, that if the trailer causes any damage to your driveway, he's responsible for rectifying it.