If you're really concerned about luggage weight, you could try looking in stores like Milletts or other camping suppliers. There are now very lightweight fibre towels available which are specifically for camping and lightweight travel which aren't very bulky but are still effective at drying.