I certainly have no problem with Elvis, if anything he did a lot of good in normalising white America’s relationship with African-American music (as did many other white musicians over the decades, Al Jolson in particular did a lot of good, especially when you consider he was around at the height of the KKK at a time when Birth of a Nation was in theaters). The alternative, given the context, was for no cultural acceptance of black art, and no progress. Cultural appropriation by those guys had an overwhelmingly positive impact on American society.
Re royalties, certainly there were a lot of people in the music business who profited off black music in an exploitative manner, but then music industry exploitation is hardly a phenomenon unique to culturally appropriated music, the problem there isn’t related to the transfer of culture, it’s orthogonal to that.