I think it's a Victorian thing. Servants couldn't be trusted to dry your precious and expensive bone china tea service with ordinary dish cloths. The head housekeeper had that responsibility and used a special linen tea towel.
The National Trust sells tea towels. Dish towel is an Americanism.