There are thousands of examples of cultural expressions that a majority of people in the culture would respect that some smaller group of people have authority over when a piece of clothing is culturally appropriate to wear.
We avoid certain clothes all the time. The issue with appropriating Native American culture is that people don’t want to give them the same consideration we give to other people every day.