I think it's important to be mindful about the intent behind the decision to adopt features of other cultures, though. Firstly, when aspects that are important to our culture are used or abused, we care about that and sometimes kick up a fuss, so it's not unreasonable when going the other way to respect that other people might be bothered if we do something similar with respect to *their* cultural icons.
Separately, one part of the "cultural appropriation" argument is related to intellectual property rights (particularly with relation to musical culture). I think this can be overdone too, but still it's not too unreasonable to be upfront about your sources and influences.
As long as the intent is respectful, there shouldn't be a problem except for those determined to find one. Still, it doesn't hurt to be more thoughtful about how and why we adopt habits and features of other cultures.